[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:10:55 dist-test-slave-t7g5 krb5kdc[3332](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:10:55 dist-test-slave-t7g5 krb5kdc[3332](info): set up 2 sockets
Aug 11 19:10:55 dist-test-slave-t7g5 krb5kdc[3332](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260811 19:10:56.355942 3318 mini_kdc.cc:121] Time spent starting KDC: real 1.390s user 0.008s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Aug 11 19:10:56 dist-test-slave-t7g5 krb5kdc[3332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475456, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:10:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:10:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.61.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab.
Entry for principal kudu/127.3.61.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab.
I20260811 19:10:56.603020 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:46771
--webserver_interface=127.3.61.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:46545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.61.190:46771
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:10:57.135721 3351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:10:57.136170 3351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:10:57.136359 3351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:10:57.146972 3351 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:10:57.147158 3351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:10:57.147255 3351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:10:57.147547 3351 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:10:57.147631 3351 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:10:57.163125 3351 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:46545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.61.190:46771
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:46771
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.61.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:10:57.164829 3351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:10:57.166561 3351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:10:57.183969 3360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:10:57.184032 3368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:10:57.183969 3363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:10:57.185777 3351 server_base.cc:1034] running on GCE node
I20260811 19:10:57.186945 3351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:10:57.188930 3351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:10:57.190318 3351 hybrid_clock.cc:648] HybridClock initialized: now 1786475457190209 us; error 72 us; skew 500 ppm
I20260811 19:10:57.190793 3351 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:10:57 dist-test-slave-t7g5 krb5kdc[3332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475457, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:10:57.195364 3351 init.cc:377] Logged in from keytab as kudu/127.3.61.190@KRBTEST.COM (short username kudu)
I20260811 19:10:57.198549 3351 webserver.cc:460] Webserver started at http://127.3.61.190:33323/ using document root <none> and password file <none>
I20260811 19:10:57.199909 3351 fs_manager.cc:359] Metadata directory not provided
I20260811 19:10:57.201084 3351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:10:57.201613 3351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:10:57.206743 3351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/master-0/data/instance:
uuid: "c05e1a395d644ac896181a38152903d0"
format_stamp: "Formatted at 2026-08-11 19:10:57 on dist-test-slave-t7g5"
server_key: "4a050eb43c7643e3fac59cad1822860b"
server_key_iv: "2b242f36e0b3aa02e951f3cfb1b6c754"
server_key_version: "encryptionkey@0"
I20260811 19:10:57.207823 3351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/master-0/wal/instance:
uuid: "c05e1a395d644ac896181a38152903d0"
format_stamp: "Formatted at 2026-08-11 19:10:57 on dist-test-slave-t7g5"
server_key: "4a050eb43c7643e3fac59cad1822860b"
server_key_iv: "2b242f36e0b3aa02e951f3cfb1b6c754"
server_key_version: "encryptionkey@0"
I20260811 19:10:57.217125 3351 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.005s sys 0.005s
I20260811 19:10:57.224320 3379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:10:57.226763 3351 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.006s sys 0.000s
I20260811 19:10:57.227123 3351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/master-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/master-0/wal
uuid: "c05e1a395d644ac896181a38152903d0"
format_stamp: "Formatted at 2026-08-11 19:10:57 on dist-test-slave-t7g5"
server_key: "4a050eb43c7643e3fac59cad1822860b"
server_key_iv: "2b242f36e0b3aa02e951f3cfb1b6c754"
server_key_version: "encryptionkey@0"
I20260811 19:10:57.227458 3351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:10:57.261831 3351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:10:57.267999 3351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:10:57.268347 3351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:10:57.302742 3351 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.190:46771
I20260811 19:10:57.302829 3469 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.190:46771 every 8 connection(s)
I20260811 19:10:57.304841 3351 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
I20260811 19:10:57.310945 3471 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:10:57.314561 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 3351
I20260811 19:10:57.314919 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/master-0/wal/instance
I20260811 19:10:57.317214 3318 external_mini_cluster.cc:1428] Setting key 602f249e165c69c9d0efb6873208ac21
I20260811 19:10:57.327930 3471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c05e1a395d644ac896181a38152903d0: Bootstrap starting.
I20260811 19:10:57.334189 3471 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c05e1a395d644ac896181a38152903d0: Neither blocks nor log segments found. Creating new log.
I20260811 19:10:57.335955 3471 log.cc:824] T 00000000000000000000000000000000 P c05e1a395d644ac896181a38152903d0: Log is configured to *not* fsync() on all Append() calls
I20260811 19:10:57.340242 3471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c05e1a395d644ac896181a38152903d0: No bootstrap required, opened a new log
I20260811 19:10:57.350199 3471 raft_consensus.cc:348] T 00000000000000000000000000000000 P c05e1a395d644ac896181a38152903d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c05e1a395d644ac896181a38152903d0" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 46771 } }
I20260811 19:10:57.350713 3471 raft_consensus.cc:374] T 00000000000000000000000000000000 P c05e1a395d644ac896181a38152903d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:10:57.350868 3471 raft_consensus.cc:729] T 00000000000000000000000000000000 P c05e1a395d644ac896181a38152903d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c05e1a395d644ac896181a38152903d0, State: Initialized, Role: FOLLOWER
I20260811 19:10:57.351817 3471 consensus_queue.cc:260] T 00000000000000000000000000000000 P c05e1a395d644ac896181a38152903d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c05e1a395d644ac896181a38152903d0" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 46771 } }
I20260811 19:10:57.352149 3471 raft_consensus.cc:388] T 00000000000000000000000000000000 P c05e1a395d644ac896181a38152903d0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:10:57.352375 3471 raft_consensus.cc:482] T 00000000000000000000000000000000 P c05e1a395d644ac896181a38152903d0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:10:57.352690 3471 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c05e1a395d644ac896181a38152903d0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:10:57.355528 3471 raft_consensus.cc:504] T 00000000000000000000000000000000 P c05e1a395d644ac896181a38152903d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c05e1a395d644ac896181a38152903d0" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 46771 } }
Aug 11 19:10:57 dist-test-slave-t7g5 krb5kdc[3332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475456, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:10:57.356303 3471 leader_election.cc:302] T 00000000000000000000000000000000 P c05e1a395d644ac896181a38152903d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c05e1a395d644ac896181a38152903d0; no voters:
I20260811 19:10:57.357256 3471 leader_election.cc:288] T 00000000000000000000000000000000 P c05e1a395d644ac896181a38152903d0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:10:57.357417 3477 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c05e1a395d644ac896181a38152903d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:10:57.359359 3477 raft_consensus.cc:686] T 00000000000000000000000000000000 P c05e1a395d644ac896181a38152903d0 [term 1 LEADER]: Becoming Leader. State: Replica: c05e1a395d644ac896181a38152903d0, State: Running, Role: LEADER
I20260811 19:10:57.360371 3477 consensus_queue.cc:237] T 00000000000000000000000000000000 P c05e1a395d644ac896181a38152903d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c05e1a395d644ac896181a38152903d0" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 46771 } }
I20260811 19:10:57.361066 3471 sys_catalog.cc:564] T 00000000000000000000000000000000 P c05e1a395d644ac896181a38152903d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:10:57.366163 3479 sys_catalog.cc:455] T 00000000000000000000000000000000 P c05e1a395d644ac896181a38152903d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c05e1a395d644ac896181a38152903d0. Latest consensus state: current_term: 1 leader_uuid: "c05e1a395d644ac896181a38152903d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c05e1a395d644ac896181a38152903d0" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 46771 } } }
I20260811 19:10:57.366988 3479 sys_catalog.cc:458] T 00000000000000000000000000000000 P c05e1a395d644ac896181a38152903d0 [sys.catalog]: This master's current role is: LEADER
I20260811 19:10:57.366206 3478 sys_catalog.cc:455] T 00000000000000000000000000000000 P c05e1a395d644ac896181a38152903d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c05e1a395d644ac896181a38152903d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c05e1a395d644ac896181a38152903d0" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 46771 } } }
I20260811 19:10:57.367759 3478 sys_catalog.cc:458] T 00000000000000000000000000000000 P c05e1a395d644ac896181a38152903d0 [sys.catalog]: This master's current role is: LEADER
I20260811 19:10:57.376122 3473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:10:57.322648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37808 (local address 127.3.61.190:46771)
0811 19:10:57.324089 (+ 1441us) server_negotiation.cc:206] Beginning negotiation
0811 19:10:57.324117 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0811 19:10:57.325433 (+ 1316us) server_negotiation.cc:407] Connection header received
0811 19:10:57.338196 (+ 12763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:10:57.338233 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:10:57.338776 (+ 543us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:10:57.339263 (+ 487us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:10:57.343959 (+ 4696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:10:57.345999 (+ 2040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:10:57.348108 (+ 2109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:10:57.348960 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:10:57.349207 (+ 247us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:10:57.358205 (+ 8998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:10:57.358256 (+ 51us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:10:57.358293 (+ 37us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:10:57.358403 (+ 110us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:10:57.364821 (+ 6418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:10:57.366389 (+ 1568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:10:57.366403 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:10:57.366412 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:10:57.366964 (+ 552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:10:57.367760 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:10:57.367766 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:10:57.367770 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:10:57.369028 (+ 1258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:10:57.369605 (+ 577us) server_negotiation.cc:1035] Waiting for connection context
0811 19:10:57.373438 (+ 3833us) server_negotiation.cc:299] Negotiation successful
0811 19:10:57.374045 (+ 607us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1005,"thread_start_us":684,"threads_started":1}
I20260811 19:10:57.387907 3485 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:10:57.395875 3485 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:10:57.408856 3485 catalog_manager.cc:1347] Generated new cluster ID: 523c25fbb9fb495a9160df6b1aae7164
I20260811 19:10:57.409009 3485 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:10:57.440836 3485 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:10:57.442528 3485 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:10:57.451828 3485 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c05e1a395d644ac896181a38152903d0: Generated new TSK 0
I20260811 19:10:57.452877 3485 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.61.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab.
Entry for principal kudu/127.3.61.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab.
I20260811 19:10:57.531610 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.129:0
--local_ip_for_outbound_sockets=127.3.61.129
--webserver_interface=127.3.61.129
--webserver_port=0
--tserver_master_addrs=127.3.61.190:46771
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab
--principal=kudu/127.3.61.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:46545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:10:58.079136 3506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:10:58.079511 3506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:10:58.079653 3506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:10:58.089325 3506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:10:58.089490 3506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:10:58.089645 3506 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.129
I20260811 19:10:58.103286 3506 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:46545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab
--principal=kudu/127.3.61.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.61.129
--webserver_port=0
--tserver_master_addrs=127.3.61.190:46771
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.129
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:10:58.104847 3506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:10:58.106372 3506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:10:58.124665 3517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:10:58.124610 3522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:10:58.124631 3516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:10:58.126654 3506 server_base.cc:1034] running on GCE node
I20260811 19:10:58.127442 3506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:10:58.128715 3506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:10:58.129998 3506 hybrid_clock.cc:648] HybridClock initialized: now 1786475458129927 us; error 38 us; skew 500 ppm
I20260811 19:10:58.130527 3506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:10:58 dist-test-slave-t7g5 krb5kdc[3332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475458, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:10:58.138705 3506 init.cc:377] Logged in from keytab as kudu/127.3.61.129@KRBTEST.COM (short username kudu)
I20260811 19:10:58.141258 3506 webserver.cc:460] Webserver started at http://127.3.61.129:42181/ using document root <none> and password file <none>
I20260811 19:10:58.142201 3506 fs_manager.cc:359] Metadata directory not provided
I20260811 19:10:58.142366 3506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:10:58.142773 3506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:10:58.146559 3506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/ts-0/data/instance:
uuid: "83a55bdcfa564130a3d521eb0915775e"
format_stamp: "Formatted at 2026-08-11 19:10:58 on dist-test-slave-t7g5"
server_key: "36af1735b32725de39f73c9b6a0a0e2c"
server_key_iv: "356674c1e4891217f10179adb006ae59"
server_key_version: "encryptionkey@0"
I20260811 19:10:58.147490 3506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/ts-0/wal/instance:
uuid: "83a55bdcfa564130a3d521eb0915775e"
format_stamp: "Formatted at 2026-08-11 19:10:58 on dist-test-slave-t7g5"
server_key: "36af1735b32725de39f73c9b6a0a0e2c"
server_key_iv: "356674c1e4891217f10179adb006ae59"
server_key_version: "encryptionkey@0"
I20260811 19:10:58.155503 3506 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260811 19:10:58.161396 3537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:10:58.163205 3506 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260811 19:10:58.163437 3506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/ts-0/wal
uuid: "83a55bdcfa564130a3d521eb0915775e"
format_stamp: "Formatted at 2026-08-11 19:10:58 on dist-test-slave-t7g5"
server_key: "36af1735b32725de39f73c9b6a0a0e2c"
server_key_iv: "356674c1e4891217f10179adb006ae59"
server_key_version: "encryptionkey@0"
I20260811 19:10:58.163709 3506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:10:58.187181 3506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:10:58.193778 3506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:10:58.194557 3506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:10:58.196923 3506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:10:58.199002 3506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:10:58.199131 3506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:10:58.199291 3506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:10:58.199357 3506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:10:58.265635 3506 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.129:38119
I20260811 19:10:58.265806 3740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.129:38119 every 8 connection(s)
I20260811 19:10:58.267128 3506 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
I20260811 19:10:58.268153 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 3506
I20260811 19:10:58.268453 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/ts-0/wal/instance
I20260811 19:10:58.269420 3318 external_mini_cluster.cc:1428] Setting key 1c853d1f990d0ff413dd16b140202406
Aug 11 19:10:58 dist-test-slave-t7g5 krb5kdc[3332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475458, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.129@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.3.61.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.130@KRBTEST.COM" created.
I20260811 19:10:58.304940 3746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:10:58.271738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:34473 (local address 127.3.61.190:46771)
0811 19:10:58.282346 (+ 10608us) server_negotiation.cc:206] Beginning negotiation
0811 19:10:58.282362 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 19:10:58.283314 (+ 952us) server_negotiation.cc:407] Connection header received
0811 19:10:58.286394 (+ 3080us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:10:58.286400 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:10:58.286510 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:10:58.286632 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:10:58.288871 (+ 2239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:10:58.290031 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:10:58.292150 (+ 2119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:10:58.292856 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:10:58.293023 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:10:58.297645 (+ 4622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:10:58.297681 (+ 36us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:10:58.297686 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:10:58.297733 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:10:58.300801 (+ 3068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:10:58.301850 (+ 1049us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:10:58.301856 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:10:58.301860 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:10:58.302038 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:10:58.302702 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:10:58.302708 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:10:58.302712 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:10:58.303118 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:10:58.303293 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 19:10:58.304278 (+ 985us) server_negotiation.cc:299] Negotiation successful
0811 19:10:58.304547 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10441,"thread_start_us":10286,"threads_started":1}
I20260811 19:10:58.306314 3747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:10:58.271694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:46771 (local address 127.3.61.129:34473)
0811 19:10:58.283082 (+ 11388us) negotiation.cc:107] Waiting for socket to connect
0811 19:10:58.283152 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0811 19:10:58.285957 (+ 2805us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:10:58.286963 (+ 1006us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:10:58.286973 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:10:58.287838 (+ 865us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:10:58.288651 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:10:58.288674 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:10:58.290222 (+ 1548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:10:58.290227 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:10:58.291967 (+ 1740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:10:58.291996 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:10:58.293236 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:10:58.293244 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:10:58.293530 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:10:58.294488 (+ 958us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:10:58.294514 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:10:58.297323 (+ 2809us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:10:58.301014 (+ 3691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:10:58.301022 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:10:58.301039 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:10:58.301687 (+ 648us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:10:58.302178 (+ 491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:10:58.302183 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:10:58.302187 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:10:58.302558 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:10:58.303273 (+ 715us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:10:58.303282 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:10:58.303763 (+ 481us) client_negotiation.cc:769] Sending connection context
0811 19:10:58.304124 (+ 361us) client_negotiation.cc:240] Negotiation successful
0811 19:10:58.304745 (+ 621us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11025,"thread_start_us":10650,"threads_started":1}
I20260811 19:10:58.308187 3741 heartbeater.cc:347] Connected to a master server at 127.3.61.190:46771
I20260811 19:10:58.308596 3741 heartbeater.cc:464] Registering TS with master...
I20260811 19:10:58.309643 3741 heartbeater.cc:511] Master 127.3.61.190:46771 requested a full tablet report, sending...
I20260811 19:10:58.313031 3409 ts_manager.cc:194] Registered new tserver with Master: 83a55bdcfa564130a3d521eb0915775e (127.3.61.129:38119)
I20260811 19:10:58.315658 3409 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.129@KRBTEST.COM'} at 127.3.61.129:34473
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab.
Entry for principal kudu/127.3.61.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab.
I20260811 19:10:58.333737 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.130:0
--local_ip_for_outbound_sockets=127.3.61.130
--webserver_interface=127.3.61.130
--webserver_port=0
--tserver_master_addrs=127.3.61.190:46771
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab
--principal=kudu/127.3.61.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:46545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:10:58.858886 3750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:10:58.859244 3750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:10:58.859407 3750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:10:58.868374 3750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:10:58.868551 3750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:10:58.868757 3750 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.130
I20260811 19:10:58.880347 3750 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:46545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab
--principal=kudu/127.3.61.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.61.130
--webserver_port=0
--tserver_master_addrs=127.3.61.190:46771
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.130
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:10:58.881861 3750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:10:58.883332 3750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:10:58.900902 3758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:10:58.901166 3763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:10:58.901736 3765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:10:58.903215 3750 server_base.cc:1034] running on GCE node
I20260811 19:10:58.903967 3750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:10:58.905103 3750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:10:58.906374 3750 hybrid_clock.cc:648] HybridClock initialized: now 1786475458906293 us; error 42 us; skew 500 ppm
I20260811 19:10:58.906752 3750 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:10:58 dist-test-slave-t7g5 krb5kdc[3332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475458, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:10:58.910866 3750 init.cc:377] Logged in from keytab as kudu/127.3.61.130@KRBTEST.COM (short username kudu)
I20260811 19:10:58.912928 3750 webserver.cc:460] Webserver started at http://127.3.61.130:38981/ using document root <none> and password file <none>
I20260811 19:10:58.913838 3750 fs_manager.cc:359] Metadata directory not provided
I20260811 19:10:58.914058 3750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:10:58.914618 3750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:10:58.918251 3750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/ts-1/data/instance:
uuid: "e6b91689872f4676be854fac9d1bb0cf"
format_stamp: "Formatted at 2026-08-11 19:10:58 on dist-test-slave-t7g5"
server_key: "bc100be6fff1d8e822c66fd3c47d2aca"
server_key_iv: "33671d2e34bcf8866163430be6fdae43"
server_key_version: "encryptionkey@0"
I20260811 19:10:58.919414 3750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/ts-1/wal/instance:
uuid: "e6b91689872f4676be854fac9d1bb0cf"
format_stamp: "Formatted at 2026-08-11 19:10:58 on dist-test-slave-t7g5"
server_key: "bc100be6fff1d8e822c66fd3c47d2aca"
server_key_iv: "33671d2e34bcf8866163430be6fdae43"
server_key_version: "encryptionkey@0"
I20260811 19:10:58.927922 3750 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.004s sys 0.004s
I20260811 19:10:58.932703 3778 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:10:58.934470 3750 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260811 19:10:58.934695 3750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/ts-1/wal
uuid: "e6b91689872f4676be854fac9d1bb0cf"
format_stamp: "Formatted at 2026-08-11 19:10:58 on dist-test-slave-t7g5"
server_key: "bc100be6fff1d8e822c66fd3c47d2aca"
server_key_iv: "33671d2e34bcf8866163430be6fdae43"
server_key_version: "encryptionkey@0"
I20260811 19:10:58.934971 3750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:10:58.953078 3750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:10:58.959606 3750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:10:58.960024 3750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:10:58.961550 3750 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:10:58.963497 3750 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:10:58.963606 3750 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:10:58.963753 3750 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:10:58.963867 3750 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:10:59.026139 3750 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.130:42883
I20260811 19:10:59.026212 3975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.130:42883 every 8 connection(s)
I20260811 19:10:59.027693 3750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
I20260811 19:10:59.032716 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 3750
I20260811 19:10:59.032987 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/ts-1/wal/instance
I20260811 19:10:59.033806 3318 external_mini_cluster.cc:1428] Setting key 963a21ccd5dbf2c208ec45f9ee5700e0
Aug 11 19:10:59 dist-test-slave-t7g5 krb5kdc[3332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475458, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.130@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:10:59.058864 3979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:10:59.031057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:59481 (local address 127.3.61.190:46771)
0811 19:10:59.031784 (+ 727us) server_negotiation.cc:206] Beginning negotiation
0811 19:10:59.031790 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:10:59.033415 (+ 1625us) server_negotiation.cc:407] Connection header received
0811 19:10:59.036514 (+ 3099us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:10:59.036522 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:10:59.036650 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:10:59.036821 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:10:59.039522 (+ 2701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:10:59.040430 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:10:59.043247 (+ 2817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:10:59.043824 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:10:59.043959 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:10:59.048982 (+ 5023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:10:59.048991 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:10:59.048996 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:10:59.049046 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:10:59.053551 (+ 4505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:10:59.054909 (+ 1358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:10:59.054917 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:10:59.054919 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:10:59.055191 (+ 272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:10:59.056001 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:10:59.056008 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:10:59.056011 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:10:59.056560 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:10:59.056769 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0811 19:10:59.058000 (+ 1231us) server_negotiation.cc:299] Negotiation successful
0811 19:10:59.058313 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":367,"threads_started":1}
I20260811 19:10:59.059914 3980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:10:59.031727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:46771 (local address 127.3.61.130:59481)
0811 19:10:59.033209 (+ 1482us) negotiation.cc:107] Waiting for socket to connect
0811 19:10:59.033265 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 19:10:59.036151 (+ 2886us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:10:59.037191 (+ 1040us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:10:59.037206 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:10:59.038184 (+ 978us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:10:59.039218 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:10:59.039253 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:10:59.040649 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:10:59.040655 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:10:59.043029 (+ 2374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:10:59.043049 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:10:59.044147 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:10:59.044156 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:10:59.044494 (+ 338us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:10:59.045334 (+ 840us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:10:59.045364 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:10:59.048595 (+ 3231us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:10:59.053826 (+ 5231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:10:59.053839 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:10:59.053861 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:10:59.054626 (+ 765us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:10:59.055378 (+ 752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:10:59.055383 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:10:59.055387 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:10:59.055764 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:10:59.056805 (+ 1041us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:10:59.056817 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:10:59.057411 (+ 594us) client_negotiation.cc:769] Sending connection context
0811 19:10:59.057818 (+ 407us) client_negotiation.cc:240] Negotiation successful
0811 19:10:59.058441 (+ 623us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1006,"thread_start_us":788,"threads_started":1}
I20260811 19:10:59.062461 3976 heartbeater.cc:347] Connected to a master server at 127.3.61.190:46771
WARNING: no policy specified for kudu/127.3.61.131@KRBTEST.COM; defaulting to no policy
I20260811 19:10:59.063117 3976 heartbeater.cc:464] Registering TS with master...
I20260811 19:10:59.064546 3976 heartbeater.cc:511] Master 127.3.61.190:46771 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.131@KRBTEST.COM" created.
I20260811 19:10:59.066991 3409 ts_manager.cc:194] Registered new tserver with Master: e6b91689872f4676be854fac9d1bb0cf (127.3.61.130:42883)
I20260811 19:10:59.068099 3409 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.130@KRBTEST.COM'} at 127.3.61.130:59481
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab.
Entry for principal kudu/127.3.61.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab.
I20260811 19:10:59.097960 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.131:0
--local_ip_for_outbound_sockets=127.3.61.131
--webserver_interface=127.3.61.131
--webserver_port=0
--tserver_master_addrs=127.3.61.190:46771
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab
--principal=kudu/127.3.61.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:46545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:10:59.320437 3741 heartbeater.cc:503] Master 127.3.61.190:46771 was elected leader, sending a full tablet report...
W20260811 19:10:59.593248 3984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:10:59.593729 3984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:10:59.593916 3984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:10:59.602730 3984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:10:59.602916 3984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:10:59.603111 3984 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.131
I20260811 19:10:59.616194 3984 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:46545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab
--principal=kudu/127.3.61.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.61.131
--webserver_port=0
--tserver_master_addrs=127.3.61.190:46771
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.131
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:10:59.618417 3984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:10:59.620227 3984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:10:59.638792 3991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:10:59.638626 3995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:10:59.638659 3992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:10:59.640601 3984 server_base.cc:1034] running on GCE node
I20260811 19:10:59.641608 3984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:10:59.643270 3984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:10:59.644593 3984 hybrid_clock.cc:648] HybridClock initialized: now 1786475459644476 us; error 77 us; skew 500 ppm
I20260811 19:10:59.645113 3984 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:10:59 dist-test-slave-t7g5 krb5kdc[3332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475459, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:10:59.649946 3984 init.cc:377] Logged in from keytab as kudu/127.3.61.131@KRBTEST.COM (short username kudu)
I20260811 19:10:59.652379 3984 webserver.cc:460] Webserver started at http://127.3.61.131:34951/ using document root <none> and password file <none>
I20260811 19:10:59.653457 3984 fs_manager.cc:359] Metadata directory not provided
I20260811 19:10:59.654752 3984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:10:59.655176 3984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:10:59.658725 3984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/ts-2/data/instance:
uuid: "c01260c6374c43978b8d89c6ad7e9943"
format_stamp: "Formatted at 2026-08-11 19:10:59 on dist-test-slave-t7g5"
server_key: "8613a3d5976ae82eedadd9b44c4b4355"
server_key_iv: "b2dda1294a1dacf6a36886fd2979b6d3"
server_key_version: "encryptionkey@0"
I20260811 19:10:59.659582 3984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/ts-2/wal/instance:
uuid: "c01260c6374c43978b8d89c6ad7e9943"
format_stamp: "Formatted at 2026-08-11 19:10:59 on dist-test-slave-t7g5"
server_key: "8613a3d5976ae82eedadd9b44c4b4355"
server_key_iv: "b2dda1294a1dacf6a36886fd2979b6d3"
server_key_version: "encryptionkey@0"
I20260811 19:10:59.667493 3984 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.005s
I20260811 19:10:59.672582 4010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:10:59.674250 3984 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260811 19:10:59.674443 3984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/ts-2/wal
uuid: "c01260c6374c43978b8d89c6ad7e9943"
format_stamp: "Formatted at 2026-08-11 19:10:59 on dist-test-slave-t7g5"
server_key: "8613a3d5976ae82eedadd9b44c4b4355"
server_key_iv: "b2dda1294a1dacf6a36886fd2979b6d3"
server_key_version: "encryptionkey@0"
I20260811 19:10:59.674715 3984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:10:59.715071 3984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:10:59.722762 3984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:10:59.723220 3984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:10:59.725147 3984 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:10:59.727458 3984 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:10:59.727603 3984 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:10:59.727775 3984 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:10:59.727838 3984 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:10:59.791877 3984 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.131:43249
I20260811 19:10:59.791965 4219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.131:43249 every 8 connection(s)
I20260811 19:10:59.793592 3984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
I20260811 19:10:59.799858 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 3984
I20260811 19:10:59.800249 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786475454804547-3318-0/minicluster-data/ts-2/wal/instance
I20260811 19:10:59.800948 3318 external_mini_cluster.cc:1428] Setting key ac3989ffbd40c204c787f39e6661697f
Aug 11 19:10:59 dist-test-slave-t7g5 krb5kdc[3332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475459, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.131@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:10:59.828429 4228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:10:59.798337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:33751 (local address 127.3.61.190:46771)
0811 19:10:59.799130 (+ 793us) server_negotiation.cc:206] Beginning negotiation
0811 19:10:59.799136 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:10:59.801385 (+ 2249us) server_negotiation.cc:407] Connection header received
0811 19:10:59.804811 (+ 3426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:10:59.804817 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:10:59.804932 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:10:59.805040 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:10:59.807326 (+ 2286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:10:59.808334 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:10:59.811008 (+ 2674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:10:59.811669 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:10:59.811850 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:10:59.817625 (+ 5775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:10:59.817635 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:10:59.817639 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:10:59.817768 (+ 129us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:10:59.823114 (+ 5346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:10:59.824828 (+ 1714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:10:59.824835 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:10:59.824837 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:10:59.825084 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:10:59.825902 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:10:59.825907 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:10:59.825909 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:10:59.826348 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:10:59.826516 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 19:10:59.827706 (+ 1190us) server_negotiation.cc:299] Negotiation successful
0811 19:10:59.827959 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":386,"threads_started":1}
I20260811 19:10:59.829813 4229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:10:59.799298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:46771 (local address 127.3.61.131:33751)
0811 19:10:59.800959 (+ 1661us) negotiation.cc:107] Waiting for socket to connect
0811 19:10:59.801035 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0811 19:10:59.804388 (+ 3353us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:10:59.805410 (+ 1022us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:10:59.805421 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:10:59.806271 (+ 850us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:10:59.807100 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:10:59.807126 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:10:59.808626 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:10:59.808633 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:10:59.810613 (+ 1980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:10:59.810656 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:10:59.812136 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:10:59.812145 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:10:59.812548 (+ 403us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:10:59.813556 (+ 1008us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:10:59.813587 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:10:59.817115 (+ 3528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:10:59.823435 (+ 6320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:10:59.823452 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:10:59.823477 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:10:59.824550 (+ 1073us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:10:59.825270 (+ 720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:10:59.825275 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:10:59.825278 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:10:59.825714 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:10:59.826641 (+ 927us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:10:59.826653 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:10:59.827192 (+ 539us) client_negotiation.cc:769] Sending connection context
0811 19:10:59.827528 (+ 336us) client_negotiation.cc:240] Negotiation successful
0811 19:10:59.828091 (+ 563us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1239,"thread_start_us":796,"threads_started":1}
I20260811 19:10:59.832445 4224 heartbeater.cc:347] Connected to a master server at 127.3.61.190:46771
I20260811 19:10:59.832981 4224 heartbeater.cc:464] Registering TS with master...
I20260811 19:10:59.834268 4224 heartbeater.cc:511] Master 127.3.61.190:46771 requested a full tablet report, sending...
I20260811 19:10:59.836709 3409 ts_manager.cc:194] Registered new tserver with Master: c01260c6374c43978b8d89c6ad7e9943 (127.3.61.131:43249)
I20260811 19:10:59.837694 3409 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.131@KRBTEST.COM'} at 127.3.61.131:33751
I20260811 19:10:59.851431 3318 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 19:10:59 dist-test-slave-t7g5 krb5kdc[3332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475459, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 19:10:59 dist-test-slave-t7g5 krb5kdc[3332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475459, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.3.61.129@KRBTEST.COM
I20260811 19:10:59.932983 4241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:10:59.907508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57406 (local address 127.3.61.129:38119)
0811 19:10:59.909436 (+ 1928us) server_negotiation.cc:206] Beginning negotiation
0811 19:10:59.909445 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:10:59.909487 (+ 42us) server_negotiation.cc:407] Connection header received
0811 19:10:59.909591 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:10:59.909600 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:10:59.909986 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:10:59.910262 (+ 276us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:10:59.912646 (+ 2384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:10:59.914044 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:10:59.915926 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:10:59.916722 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:10:59.916854 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:10:59.923654 (+ 6800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:10:59.923678 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:10:59.923698 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:10:59.923760 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:10:59.927818 (+ 4058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:10:59.928913 (+ 1095us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:10:59.928923 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:10:59.928925 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:10:59.929171 (+ 246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:10:59.930232 (+ 1061us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:10:59.930240 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:10:59.930244 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:10:59.931069 (+ 825us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:10:59.931451 (+ 382us) server_negotiation.cc:1035] Waiting for connection context
0811 19:10:59.932249 (+ 798us) server_negotiation.cc:299] Negotiation successful
0811 19:10:59.932545 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1690,"thread_start_us":1432,"threads_started":1}
W20260811 19:10:59.934855 3587 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57406
Aug 11 19:10:59 dist-test-slave-t7g5 krb5kdc[3332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475459, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 19:10:59 dist-test-slave-t7g5 krb5kdc[3332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475459, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.129@KRBTEST.COM
I20260811 19:11:00.008584 4241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:10:59.986288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57414 (local address 127.3.61.129:38119)
0811 19:10:59.986621 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0811 19:10:59.986628 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:10:59.987057 (+ 429us) server_negotiation.cc:407] Connection header received
0811 19:10:59.987341 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:10:59.987346 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:10:59.987470 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:10:59.987594 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:10:59.989767 (+ 2173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:10:59.990549 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:10:59.992347 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:10:59.992948 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:10:59.993132 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:10:59.999370 (+ 6238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:10:59.999378 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:10:59.999382 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:10:59.999449 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:00.003902 (+ 4453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:00.005370 (+ 1468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:00.005377 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:00.005379 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:00.005870 (+ 491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:00.006664 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:00.006669 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:00.006672 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:00.007092 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:00.007413 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:00.007806 (+ 393us) server_negotiation.cc:299] Negotiation successful
0811 19:11:00.008132 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134}
I20260811 19:11:00.011426 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 3506
I20260811 19:11:00.037271 4241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:00.013543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57426 (local address 127.3.61.129:38119)
0811 19:11:00.013839 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:00.013845 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:00.014118 (+ 273us) server_negotiation.cc:407] Connection header received
0811 19:11:00.014268 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:00.014273 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:00.014367 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:00.014485 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:00.016229 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:00.017080 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:00.018805 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:00.019356 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:00.019497 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:00.026913 (+ 7416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:00.026924 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:00.026929 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:00.027006 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:00.032398 (+ 5392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:00.033837 (+ 1439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:00.033844 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:00.033848 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:00.034205 (+ 357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:00.035041 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:00.035048 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:00.035050 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:00.035646 (+ 596us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:00.036106 (+ 460us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:00.036547 (+ 441us) server_negotiation.cc:299] Negotiation successful
0811 19:11:00.036793 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142}
I20260811 19:11:00.038638 3735 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:11:00.072021 3976 heartbeater.cc:503] Master 127.3.61.190:46771 was elected leader, sending a full tablet report...
I20260811 19:11:00.135782 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 3506
I20260811 19:11:00.163156 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 3750
Aug 11 19:11:00 dist-test-slave-t7g5 krb5kdc[3332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475459, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.130@KRBTEST.COM
I20260811 19:11:00.183815 4251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:00.164177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39406 (local address 127.3.61.130:42883)
0811 19:11:00.164784 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:00.164791 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:00.164823 (+ 32us) server_negotiation.cc:407] Connection header received
0811 19:11:00.164888 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:00.164896 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:00.165116 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:00.165246 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:00.166813 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:00.167885 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:00.169237 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:00.169941 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:00.170060 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:00.175527 (+ 5467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:00.175546 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:00.175565 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:00.175623 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:00.179371 (+ 3748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:00.180350 (+ 979us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:00.180359 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:00.180361 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:00.180589 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:00.181396 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:00.181403 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:00.181406 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:00.182289 (+ 883us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:00.182599 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:00.183133 (+ 534us) server_negotiation.cc:299] Negotiation successful
0811 19:11:00.183446 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":313,"threads_started":1}
I20260811 19:11:00.185986 3968 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:11:00.281877 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 3750
I20260811 19:11:00.311751 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 3984
Aug 11 19:11:00 dist-test-slave-t7g5 krb5kdc[3332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475459, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.131@KRBTEST.COM
I20260811 19:11:00.333472 4255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:00.312892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48582 (local address 127.3.61.131:43249)
0811 19:11:00.313497 (+ 605us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:00.313504 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:00.313535 (+ 31us) server_negotiation.cc:407] Connection header received
0811 19:11:00.313600 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:00.313607 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:00.313898 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:00.314029 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:00.315725 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:00.317048 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:00.319118 (+ 2070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:00.319858 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:00.319998 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:00.325294 (+ 5296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:00.325332 (+ 38us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:00.325360 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:00.325438 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:00.329290 (+ 3852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:00.330340 (+ 1050us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:00.330352 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:00.330355 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:00.330660 (+ 305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:00.331251 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:00.331259 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:00.331263 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:00.331806 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:00.332068 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:00.332803 (+ 735us) server_negotiation.cc:299] Negotiation successful
0811 19:11:00.333090 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":396,"threads_started":1}
I20260811 19:11:00.334977 4212 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:11:00.424449 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 3984
I20260811 19:11:00.447521 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 3351
I20260811 19:11:00.449329 3457 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:11:00.564895 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 3351
2026-08-11T19:11:00Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (5666 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:11:00 dist-test-slave-t7g5 krb5kdc[4265](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:11:00 dist-test-slave-t7g5 krb5kdc[4265](info): set up 2 sockets
Aug 11 19:11:00 dist-test-slave-t7g5 krb5kdc[4265](info): commencing operation
krb5kdc: starting...
W20260811 19:11:01.852047 3318 mini_kdc.cc:121] Time spent starting KDC: real 1.233s user 0.000s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Aug 11 19:11:02 dist-test-slave-t7g5 krb5kdc[4265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475462, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:11:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:11:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.61.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab.
Entry for principal kudu/127.3.61.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab.
I20260811 19:11:02.113533 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:35085
--webserver_interface=127.3.61.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:39331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.61.190:35085
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:11:02.469874 4290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:11:02.470400 4290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:11:02.470580 4290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:11:02.479278 4290 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:11:02.479398 4290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:11:02.479445 4290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:11:02.479494 4290 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:11:02.479538 4290 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:11:02.490646 4290 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:39331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.61.190:35085
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:35085
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.61.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:11:02.492368 4290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:11:02.493836 4290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:11:02.507301 4299 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:02.507280 4301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:02.507332 4307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:11:02.508661 4290 server_base.cc:1034] running on GCE node
I20260811 19:11:02.509552 4290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:11:02.511408 4290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:11:02.512868 4290 hybrid_clock.cc:648] HybridClock initialized: now 1786475462512793 us; error 58 us; skew 500 ppm
I20260811 19:11:02.513278 4290 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:11:02 dist-test-slave-t7g5 krb5kdc[4265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475462, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:11:02.517766 4290 init.cc:377] Logged in from keytab as kudu/127.3.61.190@KRBTEST.COM (short username kudu)
I20260811 19:11:02.519635 4290 webserver.cc:460] Webserver started at http://127.3.61.190:39915/ using document root <none> and password file <none>
I20260811 19:11:02.520399 4290 fs_manager.cc:359] Metadata directory not provided
I20260811 19:11:02.520529 4290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:11:02.520852 4290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:11:02.524085 4290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/master-0/data/instance:
uuid: "2ecfccea2040444ea5940f0ead1df967"
format_stamp: "Formatted at 2026-08-11 19:11:02 on dist-test-slave-t7g5"
server_key: "c79be59d0b530af85978ea08dc3465b3"
server_key_iv: "b9a913e87d8a1ac64efd582abefcffe3"
server_key_version: "encryptionkey@0"
I20260811 19:11:02.525076 4290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/master-0/wal/instance:
uuid: "2ecfccea2040444ea5940f0ead1df967"
format_stamp: "Formatted at 2026-08-11 19:11:02 on dist-test-slave-t7g5"
server_key: "c79be59d0b530af85978ea08dc3465b3"
server_key_iv: "b9a913e87d8a1ac64efd582abefcffe3"
server_key_version: "encryptionkey@0"
I20260811 19:11:02.530944 4290 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260811 19:11:02.534859 4316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:02.536300 4290 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 19:11:02.536520 4290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/master-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/master-0/wal
uuid: "2ecfccea2040444ea5940f0ead1df967"
format_stamp: "Formatted at 2026-08-11 19:11:02 on dist-test-slave-t7g5"
server_key: "c79be59d0b530af85978ea08dc3465b3"
server_key_iv: "b9a913e87d8a1ac64efd582abefcffe3"
server_key_version: "encryptionkey@0"
I20260811 19:11:02.536731 4290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:11:02.570782 4290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:11:02.576380 4290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:11:02.576828 4290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:11:02.598744 4290 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.190:35085
I20260811 19:11:02.598788 4396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.190:35085 every 8 connection(s)
I20260811 19:11:02.600195 4290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
I20260811 19:11:02.604163 4397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:11:02.609472 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 4290
I20260811 19:11:02.609730 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/master-0/wal/instance
I20260811 19:11:02.610402 3318 external_mini_cluster.cc:1428] Setting key edb1cfb7217920d27352c022f61e4f99
I20260811 19:11:02.615676 4397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ecfccea2040444ea5940f0ead1df967: Bootstrap starting.
I20260811 19:11:02.619643 4397 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ecfccea2040444ea5940f0ead1df967: Neither blocks nor log segments found. Creating new log.
I20260811 19:11:02.620945 4397 log.cc:824] T 00000000000000000000000000000000 P 2ecfccea2040444ea5940f0ead1df967: Log is configured to *not* fsync() on all Append() calls
I20260811 19:11:02.623975 4397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ecfccea2040444ea5940f0ead1df967: No bootstrap required, opened a new log
Aug 11 19:11:02 dist-test-slave-t7g5 krb5kdc[4265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475462, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:11:02.631887 4397 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ecfccea2040444ea5940f0ead1df967 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ecfccea2040444ea5940f0ead1df967" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35085 } }
I20260811 19:11:02.632289 4397 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ecfccea2040444ea5940f0ead1df967 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:11:02.632404 4397 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ecfccea2040444ea5940f0ead1df967 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ecfccea2040444ea5940f0ead1df967, State: Initialized, Role: FOLLOWER
I20260811 19:11:02.633064 4397 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ecfccea2040444ea5940f0ead1df967 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ecfccea2040444ea5940f0ead1df967" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35085 } }
I20260811 19:11:02.633386 4397 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ecfccea2040444ea5940f0ead1df967 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:11:02.633548 4397 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ecfccea2040444ea5940f0ead1df967 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:11:02.633796 4397 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ecfccea2040444ea5940f0ead1df967 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:11:02.634958 4402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:02.612628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49982 (local address 127.3.61.190:35085)
0811 19:11:02.613499 (+ 871us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:02.613512 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:02.613555 (+ 43us) server_negotiation.cc:407] Connection header received
0811 19:11:02.615337 (+ 1782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:02.615360 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:02.615769 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:02.616114 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:02.618055 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:02.619427 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:02.621188 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:02.622049 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:02.622247 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:02.626635 (+ 4388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:02.626644 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:02.626661 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:02.626712 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:02.630100 (+ 3388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:02.630806 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:02.630817 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:02.630827 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:02.631016 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:02.631515 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:02.631519 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:02.631521 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:02.632107 (+ 586us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:02.632431 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:02.632984 (+ 553us) server_negotiation.cc:299] Negotiation successful
0811 19:11:02.633473 (+ 489us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":628,"thread_start_us":381,"threads_started":1}
I20260811 19:11:02.636925 4397 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ecfccea2040444ea5940f0ead1df967 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ecfccea2040444ea5940f0ead1df967" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35085 } }
I20260811 19:11:02.638227 4397 leader_election.cc:302] T 00000000000000000000000000000000 P 2ecfccea2040444ea5940f0ead1df967 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ecfccea2040444ea5940f0ead1df967; no voters:
I20260811 19:11:02.639086 4397 leader_election.cc:288] T 00000000000000000000000000000000 P 2ecfccea2040444ea5940f0ead1df967 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:11:02.639225 4407 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ecfccea2040444ea5940f0ead1df967 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:11:02.640030 4407 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ecfccea2040444ea5940f0ead1df967 [term 1 LEADER]: Becoming Leader. State: Replica: 2ecfccea2040444ea5940f0ead1df967, State: Running, Role: LEADER
I20260811 19:11:02.640686 4407 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ecfccea2040444ea5940f0ead1df967 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ecfccea2040444ea5940f0ead1df967" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35085 } }
I20260811 19:11:02.641517 4397 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ecfccea2040444ea5940f0ead1df967 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:11:02.643762 4410 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ecfccea2040444ea5940f0ead1df967 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ecfccea2040444ea5940f0ead1df967. Latest consensus state: current_term: 1 leader_uuid: "2ecfccea2040444ea5940f0ead1df967" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ecfccea2040444ea5940f0ead1df967" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35085 } } }
I20260811 19:11:02.644164 4410 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ecfccea2040444ea5940f0ead1df967 [sys.catalog]: This master's current role is: LEADER
I20260811 19:11:02.645205 4409 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ecfccea2040444ea5940f0ead1df967 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ecfccea2040444ea5940f0ead1df967" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ecfccea2040444ea5940f0ead1df967" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35085 } } }
I20260811 19:11:02.645586 4409 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ecfccea2040444ea5940f0ead1df967 [sys.catalog]: This master's current role is: LEADER
I20260811 19:11:02.647289 4422 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:11:02.653467 4422 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:11:02.664561 4422 catalog_manager.cc:1347] Generated new cluster ID: 3af4fd078a414c59964e811986137a6e
I20260811 19:11:02.664721 4422 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:11:02.679941 4422 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:11:02.682518 4422 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:11:02.691356 4422 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ecfccea2040444ea5940f0ead1df967: Generated new TSK 0
I20260811 19:11:02.692780 4422 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.61.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab.
Entry for principal kudu/127.3.61.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab.
I20260811 19:11:02.758569 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.129:0
--local_ip_for_outbound_sockets=127.3.61.129
--webserver_interface=127.3.61.129
--webserver_port=0
--tserver_master_addrs=127.3.61.190:35085
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab
--principal=kudu/127.3.61.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:39331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:11:03.242261 4452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:11:03.242759 4452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:11:03.243053 4452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:11:03.257225 4452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:11:03.257565 4452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:11:03.257895 4452 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.129
I20260811 19:11:03.277972 4452 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:39331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab
--principal=kudu/127.3.61.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.61.129
--webserver_port=0
--tserver_master_addrs=127.3.61.190:35085
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.129
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:11:03.280109 4452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:11:03.282027 4452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:11:03.300415 4460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:03.300354 4461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:03.300534 4467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:11:03.302793 4452 server_base.cc:1034] running on GCE node
I20260811 19:11:03.303777 4452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:11:03.304996 4452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:11:03.306278 4452 hybrid_clock.cc:648] HybridClock initialized: now 1786475463306186 us; error 50 us; skew 500 ppm
I20260811 19:11:03.306686 4452 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:11:03 dist-test-slave-t7g5 krb5kdc[4265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475463, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:11:03.310755 4452 init.cc:377] Logged in from keytab as kudu/127.3.61.129@KRBTEST.COM (short username kudu)
I20260811 19:11:03.312731 4452 webserver.cc:460] Webserver started at http://127.3.61.129:36963/ using document root <none> and password file <none>
I20260811 19:11:03.313578 4452 fs_manager.cc:359] Metadata directory not provided
I20260811 19:11:03.313778 4452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:11:03.314158 4452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:11:03.318878 4452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/ts-0/data/instance:
uuid: "ae4773cc6b924efbbcefec0f1f0c0077"
format_stamp: "Formatted at 2026-08-11 19:11:03 on dist-test-slave-t7g5"
server_key: "687b863964b63342bbe62aa6c0955db0"
server_key_iv: "1bf5d1484086ee10ff300702f7b34b5a"
server_key_version: "encryptionkey@0"
I20260811 19:11:03.320142 4452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/ts-0/wal/instance:
uuid: "ae4773cc6b924efbbcefec0f1f0c0077"
format_stamp: "Formatted at 2026-08-11 19:11:03 on dist-test-slave-t7g5"
server_key: "687b863964b63342bbe62aa6c0955db0"
server_key_iv: "1bf5d1484086ee10ff300702f7b34b5a"
server_key_version: "encryptionkey@0"
I20260811 19:11:03.327577 4452 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.001s sys 0.007s
I20260811 19:11:03.332296 4477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:03.334530 4452 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 19:11:03.334756 4452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/ts-0/wal
uuid: "ae4773cc6b924efbbcefec0f1f0c0077"
format_stamp: "Formatted at 2026-08-11 19:11:03 on dist-test-slave-t7g5"
server_key: "687b863964b63342bbe62aa6c0955db0"
server_key_iv: "1bf5d1484086ee10ff300702f7b34b5a"
server_key_version: "encryptionkey@0"
I20260811 19:11:03.335021 4452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:11:03.351399 4452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:11:03.357702 4452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:11:03.358122 4452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:11:03.359293 4452 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:11:03.361181 4452 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:11:03.361281 4452 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:03.361521 4452 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:11:03.361682 4452 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:03.448002 4452 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.129:44483
I20260811 19:11:03.448043 4684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.129:44483 every 8 connection(s)
I20260811 19:11:03.449900 4452 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
I20260811 19:11:03.459448 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 4452
I20260811 19:11:03.459848 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/ts-0/wal/instance
I20260811 19:11:03.460705 3318 external_mini_cluster.cc:1428] Setting key 4251ac134e9c196891cc008ceabf779a
Aug 11 19:11:03 dist-test-slave-t7g5 krb5kdc[4265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475463, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.129@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:11:03.485948 4688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:03.454744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:37331 (local address 127.3.61.190:35085)
0811 19:11:03.455666 (+ 922us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:03.455672 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:03.457753 (+ 2081us) server_negotiation.cc:407] Connection header received
0811 19:11:03.461413 (+ 3660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:03.461420 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:03.461513 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:03.461612 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:03.464716 (+ 3104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:03.465863 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:03.468523 (+ 2660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:03.469244 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:03.469399 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:03.475864 (+ 6465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:03.475870 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:03.475873 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:03.475914 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:03.480429 (+ 4515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:03.482073 (+ 1644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:03.482079 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:03.482082 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:03.482382 (+ 300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:03.483340 (+ 958us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:03.483345 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:03.483348 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:03.483827 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:03.483986 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:03.485032 (+ 1046us) server_negotiation.cc:299] Negotiation successful
0811 19:11:03.485295 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":809,"thread_start_us":605,"threads_started":1}
WARNING: no policy specified for kudu/127.3.61.130@KRBTEST.COM; defaulting to no policy
I20260811 19:11:03.487298 4690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:03.455705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35085 (local address 127.3.61.129:37331)
0811 19:11:03.457552 (+ 1847us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:03.457598 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:03.461063 (+ 3465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:03.462043 (+ 980us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:03.462058 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:03.463314 (+ 1256us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:11:03.464501 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:03.464530 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:03.466074 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:03.466079 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:03.468356 (+ 2277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:03.468371 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:03.469743 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:03.469752 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:03.470142 (+ 390us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:03.471103 (+ 961us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:11:03.471136 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:11:03.475507 (+ 4371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:11:03.480671 (+ 5164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:03.480681 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:03.480699 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:03.481841 (+ 1142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:03.482536 (+ 695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:03.482542 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:03.482545 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:03.483132 (+ 587us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:03.483987 (+ 855us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:11:03.483997 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:11:03.484512 (+ 515us) client_negotiation.cc:769] Sending connection context
0811 19:11:03.484793 (+ 281us) client_negotiation.cc:240] Negotiation successful
0811 19:11:03.485541 (+ 748us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1507,"thread_start_us":1248,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.130@KRBTEST.COM" created.
I20260811 19:11:03.489928 4685 heartbeater.cc:347] Connected to a master server at 127.3.61.190:35085
I20260811 19:11:03.490588 4685 heartbeater.cc:464] Registering TS with master...
I20260811 19:11:03.492484 4685 heartbeater.cc:511] Master 127.3.61.190:35085 requested a full tablet report, sending...
I20260811 19:11:03.495743 4341 ts_manager.cc:194] Registered new tserver with Master: ae4773cc6b924efbbcefec0f1f0c0077 (127.3.61.129:44483)
I20260811 19:11:03.498695 4341 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.129@KRBTEST.COM'} at 127.3.61.129:37331
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab.
Entry for principal kudu/127.3.61.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab.
I20260811 19:11:03.527334 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.130:0
--local_ip_for_outbound_sockets=127.3.61.130
--webserver_interface=127.3.61.130
--webserver_port=0
--tserver_master_addrs=127.3.61.190:35085
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab
--principal=kudu/127.3.61.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:39331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:11:04.015990 4693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:11:04.016322 4693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:11:04.016434 4693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:11:04.025292 4693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:11:04.025454 4693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:11:04.025632 4693 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.130
I20260811 19:11:04.037372 4693 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:39331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab
--principal=kudu/127.3.61.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.61.130
--webserver_port=0
--tserver_master_addrs=127.3.61.190:35085
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.130
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:11:04.038955 4693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:11:04.040488 4693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:11:04.059029 4708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:04.058946 4703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:04.059417 4704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:11:04.060883 4693 server_base.cc:1034] running on GCE node
I20260811 19:11:04.061635 4693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:11:04.062960 4693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:11:04.064291 4693 hybrid_clock.cc:648] HybridClock initialized: now 1786475464064225 us; error 67 us; skew 500 ppm
I20260811 19:11:04.064699 4693 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:11:04 dist-test-slave-t7g5 krb5kdc[4265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475464, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:11:04.068531 4693 init.cc:377] Logged in from keytab as kudu/127.3.61.130@KRBTEST.COM (short username kudu)
I20260811 19:11:04.070430 4693 webserver.cc:460] Webserver started at http://127.3.61.130:38209/ using document root <none> and password file <none>
I20260811 19:11:04.071233 4693 fs_manager.cc:359] Metadata directory not provided
I20260811 19:11:04.071354 4693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:11:04.071694 4693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:11:04.074887 4693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/ts-1/data/instance:
uuid: "ca35ccb669e445c795c89d47d72466a9"
format_stamp: "Formatted at 2026-08-11 19:11:04 on dist-test-slave-t7g5"
server_key: "762ca68d2a69baa3ed15949dcb765b77"
server_key_iv: "c81230fec05bee2bb8160d45ae1ab838"
server_key_version: "encryptionkey@0"
I20260811 19:11:04.075793 4693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/ts-1/wal/instance:
uuid: "ca35ccb669e445c795c89d47d72466a9"
format_stamp: "Formatted at 2026-08-11 19:11:04 on dist-test-slave-t7g5"
server_key: "762ca68d2a69baa3ed15949dcb765b77"
server_key_iv: "c81230fec05bee2bb8160d45ae1ab838"
server_key_version: "encryptionkey@0"
I20260811 19:11:04.082351 4693 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 19:11:04.086182 4720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:04.087909 4693 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 19:11:04.088093 4693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/ts-1/wal
uuid: "ca35ccb669e445c795c89d47d72466a9"
format_stamp: "Formatted at 2026-08-11 19:11:04 on dist-test-slave-t7g5"
server_key: "762ca68d2a69baa3ed15949dcb765b77"
server_key_iv: "c81230fec05bee2bb8160d45ae1ab838"
server_key_version: "encryptionkey@0"
I20260811 19:11:04.088316 4693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:11:04.113355 4693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:11:04.119270 4693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:11:04.119576 4693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:11:04.121065 4693 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:11:04.122782 4693 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:11:04.122867 4693 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:04.123001 4693 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:11:04.123054 4693 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:04.161042 4693 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.130:34029
I20260811 19:11:04.161075 4920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.130:34029 every 8 connection(s)
I20260811 19:11:04.162436 4693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
I20260811 19:11:04.169090 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 4693
I20260811 19:11:04.169317 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/ts-1/wal/instance
I20260811 19:11:04.169899 3318 external_mini_cluster.cc:1428] Setting key 5c068ca700439089c73fbeb7e15c715d
Aug 11 19:11:04 dist-test-slave-t7g5 krb5kdc[4265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475464, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.130@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:11:04.185781 4925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:04.165312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:55591 (local address 127.3.61.190:35085)
0811 19:11:04.165775 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:04.165781 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:04.167283 (+ 1502us) server_negotiation.cc:407] Connection header received
0811 19:11:04.169909 (+ 2626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:04.169914 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:04.170006 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:04.170137 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:04.172026 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:04.172762 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:04.174393 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:04.174828 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:04.174946 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:04.178914 (+ 3968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:04.178919 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:04.178923 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:04.178960 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:04.182125 (+ 3165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:04.183058 (+ 933us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:04.183064 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:04.183067 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:04.183234 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:04.183753 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:04.183758 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:04.183761 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:04.184121 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:04.184271 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:04.185154 (+ 883us) server_negotiation.cc:299] Negotiation successful
0811 19:11:04.185413 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":235,"threads_started":1}
I20260811 19:11:04.186676 4926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:04.165737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35085 (local address 127.3.61.130:55591)
0811 19:11:04.166969 (+ 1232us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:04.167057 (+ 88us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:04.169441 (+ 2384us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:04.170433 (+ 992us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:04.170442 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:04.171142 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:11:04.171856 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:04.171881 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:04.172908 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:04.172912 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:04.174239 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:04.174252 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:04.175061 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:04.175069 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:04.175292 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:04.175987 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:11:04.176013 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:11:04.178652 (+ 2639us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:11:04.182313 (+ 3661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:04.182321 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:04.182335 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:04.182899 (+ 564us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:04.183378 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:04.183383 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:04.183385 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:04.183614 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:04.184264 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:11:04.184270 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:11:04.184733 (+ 463us) client_negotiation.cc:769] Sending connection context
0811 19:11:04.184992 (+ 259us) client_negotiation.cc:240] Negotiation successful
0811 19:11:04.185437 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":333,"threads_started":1}
I20260811 19:11:04.188308 4922 heartbeater.cc:347] Connected to a master server at 127.3.61.190:35085
I20260811 19:11:04.188674 4922 heartbeater.cc:464] Registering TS with master...
I20260811 19:11:04.189627 4922 heartbeater.cc:511] Master 127.3.61.190:35085 requested a full tablet report, sending...
I20260811 19:11:04.191634 4341 ts_manager.cc:194] Registered new tserver with Master: ca35ccb669e445c795c89d47d72466a9 (127.3.61.130:34029)
WARNING: no policy specified for kudu/127.3.61.131@KRBTEST.COM; defaulting to no policy
I20260811 19:11:04.193496 4341 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.130@KRBTEST.COM'} at 127.3.61.130:55591
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab.
Entry for principal kudu/127.3.61.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab.
I20260811 19:11:04.225771 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.131:0
--local_ip_for_outbound_sockets=127.3.61.131
--webserver_interface=127.3.61.131
--webserver_port=0
--tserver_master_addrs=127.3.61.190:35085
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab
--principal=kudu/127.3.61.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:39331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:11:04.504460 4685 heartbeater.cc:503] Master 127.3.61.190:35085 was elected leader, sending a full tablet report...
W20260811 19:11:04.582429 4933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:11:04.582769 4933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:11:04.582927 4933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:11:04.591372 4933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:11:04.591527 4933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:11:04.591704 4933 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.131
I20260811 19:11:04.605607 4933 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:39331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab
--principal=kudu/127.3.61.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.61.131
--webserver_port=0
--tserver_master_addrs=127.3.61.190:35085
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.131
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:11:04.607328 4933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:11:04.609055 4933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:11:04.622534 4938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:04.622609 4941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:04.622741 4939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:11:04.623805 4933 server_base.cc:1034] running on GCE node
I20260811 19:11:04.624454 4933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:11:04.625494 4933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:11:04.626735 4933 hybrid_clock.cc:648] HybridClock initialized: now 1786475464626677 us; error 70 us; skew 500 ppm
I20260811 19:11:04.627092 4933 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:11:04 dist-test-slave-t7g5 krb5kdc[4265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475464, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:11:04.630457 4933 init.cc:377] Logged in from keytab as kudu/127.3.61.131@KRBTEST.COM (short username kudu)
I20260811 19:11:04.632154 4933 webserver.cc:460] Webserver started at http://127.3.61.131:42877/ using document root <none> and password file <none>
I20260811 19:11:04.632898 4933 fs_manager.cc:359] Metadata directory not provided
I20260811 19:11:04.633013 4933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:11:04.633334 4933 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:11:04.636492 4933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/ts-2/data/instance:
uuid: "9562c35b63a44110baa2927f0aebd3ea"
format_stamp: "Formatted at 2026-08-11 19:11:04 on dist-test-slave-t7g5"
server_key: "3bb2ee276da590bb850362d0ff485d0b"
server_key_iv: "b3828745ad048913a81e1be58bc3496e"
server_key_version: "encryptionkey@0"
I20260811 19:11:04.637331 4933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/ts-2/wal/instance:
uuid: "9562c35b63a44110baa2927f0aebd3ea"
format_stamp: "Formatted at 2026-08-11 19:11:04 on dist-test-slave-t7g5"
server_key: "3bb2ee276da590bb850362d0ff485d0b"
server_key_iv: "b3828745ad048913a81e1be58bc3496e"
server_key_version: "encryptionkey@0"
I20260811 19:11:04.643252 4933 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 19:11:04.648056 4952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:04.649689 4933 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260811 19:11:04.650236 4933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/ts-2/wal
uuid: "9562c35b63a44110baa2927f0aebd3ea"
format_stamp: "Formatted at 2026-08-11 19:11:04 on dist-test-slave-t7g5"
server_key: "3bb2ee276da590bb850362d0ff485d0b"
server_key_iv: "b3828745ad048913a81e1be58bc3496e"
server_key_version: "encryptionkey@0"
I20260811 19:11:04.650708 4933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:11:04.666782 4933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:11:04.671293 4933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:11:04.671700 4933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:11:04.672775 4933 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:11:04.674268 4933 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:11:04.674350 4933 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:04.674479 4933 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:11:04.674539 4933 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:04.709000 4933 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.131:40467
I20260811 19:11:04.709084 5143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.131:40467 every 8 connection(s)
I20260811 19:11:04.710348 4933 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
I20260811 19:11:04.720808 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 4933
I20260811 19:11:04.721045 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786475454804547-3318-0/minicluster-data/ts-2/wal/instance
I20260811 19:11:04.721483 3318 external_mini_cluster.cc:1428] Setting key 1198c40d478fba91af2948fad5627721
Aug 11 19:11:04 dist-test-slave-t7g5 krb5kdc[4265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475464, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.131@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:11:04.732522 5147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:04.713147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:36897 (local address 127.3.61.190:35085)
0811 19:11:04.713622 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:04.713627 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:04.714571 (+ 944us) server_negotiation.cc:407] Connection header received
0811 19:11:04.716600 (+ 2029us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:04.716633 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:04.716721 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:04.716816 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:04.718393 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:04.719175 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:04.720917 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:04.721379 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:04.721745 (+ 366us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:04.725645 (+ 3900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:04.725676 (+ 31us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:04.725681 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:04.725724 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:04.729017 (+ 3293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:04.730065 (+ 1048us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:04.730070 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:04.730072 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:04.730211 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:04.730655 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:04.730659 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:04.730662 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:04.730968 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:04.731088 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:04.732047 (+ 959us) server_negotiation.cc:299] Negotiation successful
0811 19:11:04.732229 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":227,"threads_started":1}
I20260811 19:11:04.733552 5148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:04.713634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35085 (local address 127.3.61.131:36897)
0811 19:11:04.714410 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:04.714455 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:04.716333 (+ 1878us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:04.717063 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:04.717072 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:04.717613 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:11:04.718222 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:04.718240 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:04.719329 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:04.719333 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:04.720741 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:04.720758 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:04.721634 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:04.721643 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:04.721972 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:04.722696 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:11:04.722722 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:11:04.725434 (+ 2712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:11:04.729233 (+ 3799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:04.729245 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:04.729266 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:04.729911 (+ 645us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:04.730316 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:04.730328 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:04.730331 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:04.730538 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:04.731129 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:11:04.731137 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:11:04.731623 (+ 486us) client_negotiation.cc:769] Sending connection context
0811 19:11:04.731869 (+ 246us) client_negotiation.cc:240] Negotiation successful
0811 19:11:04.732315 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":294,"threads_started":1}
I20260811 19:11:04.735178 5144 heartbeater.cc:347] Connected to a master server at 127.3.61.190:35085
I20260811 19:11:04.735550 5144 heartbeater.cc:464] Registering TS with master...
I20260811 19:11:04.736614 5144 heartbeater.cc:511] Master 127.3.61.190:35085 requested a full tablet report, sending...
I20260811 19:11:04.738508 4341 ts_manager.cc:194] Registered new tserver with Master: 9562c35b63a44110baa2927f0aebd3ea (127.3.61.131:40467)
I20260811 19:11:04.739387 4341 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.131@KRBTEST.COM'} at 127.3.61.131:36897
I20260811 19:11:04.751456 3318 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 19:11:04.772181 5147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:04.758641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51550 (local address 127.3.61.190:35085)
0811 19:11:04.758978 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:04.758986 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:04.759547 (+ 561us) server_negotiation.cc:407] Connection header received
0811 19:11:04.759832 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:04.759836 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:04.759912 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:04.760000 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:04.761526 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:04.762319 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:04.763566 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:04.764009 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:04.764105 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:04.766349 (+ 2244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:04.766355 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:04.766358 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:04.766394 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:04.769106 (+ 2712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:04.769948 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:04.769962 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:04.769965 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:04.770122 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:04.770709 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:04.770714 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:04.770717 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:04.771099 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:04.771313 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:04.771749 (+ 436us) server_negotiation.cc:299] Negotiation successful
0811 19:11:04.771949 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
I20260811 19:11:04.780009 4341 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51550:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 19:11:04.787397 4341 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:11:04.832516 5165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:04.802030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44616 (local address 127.3.61.131:40467)
0811 19:11:04.824021 (+ 21991us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:04.824029 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:04.824064 (+ 35us) server_negotiation.cc:407] Connection header received
0811 19:11:04.824163 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:04.824169 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:04.824363 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:04.824766 (+ 403us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:11:04.825385 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:04.827061 (+ 1676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:04.830053 (+ 2992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:04.831016 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:04.831140 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:04.831510 (+ 370us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:04.831921 (+ 411us) server_negotiation.cc:299] Negotiation successful
0811 19:11:04.832099 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21808,"thread_start_us":21635,"threads_started":1}
I20260811 19:11:04.832545 5167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:04.802428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.130:34029 (local address 127.0.0.1:41128)
0811 19:11:04.823598 (+ 21170us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:04.823619 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:04.823929 (+ 310us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:04.824853 (+ 924us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:04.824857 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:04.824881 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:11:04.825438 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:04.825455 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:04.827521 (+ 2066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:04.827525 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:04.829695 (+ 2170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:04.829710 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:04.831678 (+ 1968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:04.831690 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:04.831889 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:04.831925 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 19:11:04.832015 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 19:11:04.832155 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21050,"mutex_wait_us":8,"thread_start_us":20854,"threads_started":1}
I20260811 19:11:04.833415 5168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:04.802815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52118 (local address 127.3.61.129:44483)
0811 19:11:04.822701 (+ 19886us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:04.822711 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:04.823497 (+ 786us) server_negotiation.cc:407] Connection header received
0811 19:11:04.823615 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:04.823624 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:04.823922 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:04.824159 (+ 237us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:11:04.825817 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:04.826834 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:04.829889 (+ 3055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:04.831328 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:04.831557 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:04.831784 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:04.832929 (+ 1145us) server_negotiation.cc:299] Negotiation successful
0811 19:11:04.833101 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19655,"thread_start_us":19453,"threads_started":1}
I20260811 19:11:04.833539 5164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:04.802108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41128 (local address 127.3.61.130:34029)
0811 19:11:04.813352 (+ 11244us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:04.813364 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:04.823804 (+ 10440us) server_negotiation.cc:407] Connection header received
0811 19:11:04.824119 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:04.824127 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:04.824362 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:04.824549 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:11:04.825686 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:04.827247 (+ 1561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:04.829865 (+ 2618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:04.831329 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:04.831558 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:04.831753 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:04.833065 (+ 1312us) server_negotiation.cc:299] Negotiation successful
0811 19:11:04.833238 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11014,"thread_start_us":10837,"threads_started":1}
I20260811 19:11:04.832767 5166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:04.802207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:44483 (local address 127.0.0.1:52118)
0811 19:11:04.823176 (+ 20969us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:04.823197 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:04.823343 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:04.824470 (+ 1127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:04.824477 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:04.824510 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:11:04.825626 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:04.825637 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:04.826965 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:04.826970 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:04.829692 (+ 2722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:04.829709 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:04.831931 (+ 2222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:04.831938 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:04.832102 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:04.832138 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 19:11:04.832218 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 19:11:04.832346 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20832,"mutex_wait_us":1,"spinlock_wait_cycles":7424,"thread_start_us":20682,"threads_started":1}
I20260811 19:11:04.833410 5163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:04.801740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.131:40467 (local address 127.0.0.1:44616)
0811 19:11:04.820483 (+ 18743us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:04.820526 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:04.820815 (+ 289us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:04.824575 (+ 3760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:04.824579 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:04.824609 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:11:04.825181 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:04.825201 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:04.827414 (+ 2213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:04.827418 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:04.829903 (+ 2485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:04.829914 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:04.831204 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:04.831210 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:04.831493 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:04.831560 (+ 67us) client_negotiation.cc:769] Sending connection context
0811 19:11:04.831717 (+ 157us) client_negotiation.cc:240] Negotiation successful
0811 19:11:04.831986 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18449,"mutex_wait_us":113,"spinlock_wait_cycles":768,"thread_start_us":18275,"threads_started":1}
I20260811 19:11:04.837517 4805 tablet_service.cc:1467] Processing CreateTablet for tablet cde62da5febe4e4197881f5c79368708 (DEFAULT_TABLE table=test-table [id=b6a1607ac9604286902469806e6671f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:11:04.838135 5030 tablet_service.cc:1467] Processing CreateTablet for tablet cde62da5febe4e4197881f5c79368708 (DEFAULT_TABLE table=test-table [id=b6a1607ac9604286902469806e6671f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:11:04.837694 4567 tablet_service.cc:1467] Processing CreateTablet for tablet cde62da5febe4e4197881f5c79368708 (DEFAULT_TABLE table=test-table [id=b6a1607ac9604286902469806e6671f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:11:04.839219 4805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cde62da5febe4e4197881f5c79368708. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:11:04.839609 4567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cde62da5febe4e4197881f5c79368708. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:11:04.839607 5030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cde62da5febe4e4197881f5c79368708. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:11:04.852421 5170 tablet_bootstrap.cc:492] T cde62da5febe4e4197881f5c79368708 P ca35ccb669e445c795c89d47d72466a9: Bootstrap starting.
I20260811 19:11:04.856371 5170 tablet_bootstrap.cc:654] T cde62da5febe4e4197881f5c79368708 P ca35ccb669e445c795c89d47d72466a9: Neither blocks nor log segments found. Creating new log.
I20260811 19:11:04.857455 5170 log.cc:824] T cde62da5febe4e4197881f5c79368708 P ca35ccb669e445c795c89d47d72466a9: Log is configured to *not* fsync() on all Append() calls
I20260811 19:11:04.858456 5172 tablet_bootstrap.cc:492] T cde62da5febe4e4197881f5c79368708 P 9562c35b63a44110baa2927f0aebd3ea: Bootstrap starting.
I20260811 19:11:04.860486 5173 tablet_bootstrap.cc:492] T cde62da5febe4e4197881f5c79368708 P ae4773cc6b924efbbcefec0f1f0c0077: Bootstrap starting.
I20260811 19:11:04.862470 5172 tablet_bootstrap.cc:654] T cde62da5febe4e4197881f5c79368708 P 9562c35b63a44110baa2927f0aebd3ea: Neither blocks nor log segments found. Creating new log.
I20260811 19:11:04.863349 5170 tablet_bootstrap.cc:492] T cde62da5febe4e4197881f5c79368708 P ca35ccb669e445c795c89d47d72466a9: No bootstrap required, opened a new log
I20260811 19:11:04.863621 5170 ts_tablet_manager.cc:1397] T cde62da5febe4e4197881f5c79368708 P ca35ccb669e445c795c89d47d72466a9: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20260811 19:11:04.863762 5172 log.cc:824] T cde62da5febe4e4197881f5c79368708 P 9562c35b63a44110baa2927f0aebd3ea: Log is configured to *not* fsync() on all Append() calls
I20260811 19:11:04.864025 5173 tablet_bootstrap.cc:654] T cde62da5febe4e4197881f5c79368708 P ae4773cc6b924efbbcefec0f1f0c0077: Neither blocks nor log segments found. Creating new log.
I20260811 19:11:04.865123 5173 log.cc:824] T cde62da5febe4e4197881f5c79368708 P ae4773cc6b924efbbcefec0f1f0c0077: Log is configured to *not* fsync() on all Append() calls
I20260811 19:11:04.867077 5172 tablet_bootstrap.cc:492] T cde62da5febe4e4197881f5c79368708 P 9562c35b63a44110baa2927f0aebd3ea: No bootstrap required, opened a new log
I20260811 19:11:04.867360 5172 ts_tablet_manager.cc:1397] T cde62da5febe4e4197881f5c79368708 P 9562c35b63a44110baa2927f0aebd3ea: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 19:11:04.868021 5173 tablet_bootstrap.cc:492] T cde62da5febe4e4197881f5c79368708 P ae4773cc6b924efbbcefec0f1f0c0077: No bootstrap required, opened a new log
I20260811 19:11:04.868270 5173 ts_tablet_manager.cc:1397] T cde62da5febe4e4197881f5c79368708 P ae4773cc6b924efbbcefec0f1f0c0077: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260811 19:11:04.872570 5170 raft_consensus.cc:348] T cde62da5febe4e4197881f5c79368708 P ca35ccb669e445c795c89d47d72466a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9562c35b63a44110baa2927f0aebd3ea" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 40467 } } peers { permanent_uuid: "ca35ccb669e445c795c89d47d72466a9" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 34029 } } peers { permanent_uuid: "ae4773cc6b924efbbcefec0f1f0c0077" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 44483 } }
I20260811 19:11:04.872928 5170 raft_consensus.cc:374] T cde62da5febe4e4197881f5c79368708 P ca35ccb669e445c795c89d47d72466a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:11:04.873019 5170 raft_consensus.cc:729] T cde62da5febe4e4197881f5c79368708 P ca35ccb669e445c795c89d47d72466a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca35ccb669e445c795c89d47d72466a9, State: Initialized, Role: FOLLOWER
I20260811 19:11:04.873561 5170 consensus_queue.cc:260] T cde62da5febe4e4197881f5c79368708 P ca35ccb669e445c795c89d47d72466a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9562c35b63a44110baa2927f0aebd3ea" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 40467 } } peers { permanent_uuid: "ca35ccb669e445c795c89d47d72466a9" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 34029 } } peers { permanent_uuid: "ae4773cc6b924efbbcefec0f1f0c0077" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 44483 } }
I20260811 19:11:04.874703 4922 heartbeater.cc:503] Master 127.3.61.190:35085 was elected leader, sending a full tablet report...
I20260811 19:11:04.875155 5170 ts_tablet_manager.cc:1428] T cde62da5febe4e4197881f5c79368708 P ca35ccb669e445c795c89d47d72466a9: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260811 19:11:04.878376 5173 raft_consensus.cc:348] T cde62da5febe4e4197881f5c79368708 P ae4773cc6b924efbbcefec0f1f0c0077 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9562c35b63a44110baa2927f0aebd3ea" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 40467 } } peers { permanent_uuid: "ca35ccb669e445c795c89d47d72466a9" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 34029 } } peers { permanent_uuid: "ae4773cc6b924efbbcefec0f1f0c0077" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 44483 } }
I20260811 19:11:04.878815 5173 raft_consensus.cc:374] T cde62da5febe4e4197881f5c79368708 P ae4773cc6b924efbbcefec0f1f0c0077 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:11:04.878925 5173 raft_consensus.cc:729] T cde62da5febe4e4197881f5c79368708 P ae4773cc6b924efbbcefec0f1f0c0077 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae4773cc6b924efbbcefec0f1f0c0077, State: Initialized, Role: FOLLOWER
I20260811 19:11:04.879576 5173 consensus_queue.cc:260] T cde62da5febe4e4197881f5c79368708 P ae4773cc6b924efbbcefec0f1f0c0077 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9562c35b63a44110baa2927f0aebd3ea" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 40467 } } peers { permanent_uuid: "ca35ccb669e445c795c89d47d72466a9" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 34029 } } peers { permanent_uuid: "ae4773cc6b924efbbcefec0f1f0c0077" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 44483 } }
I20260811 19:11:04.880164 5172 raft_consensus.cc:348] T cde62da5febe4e4197881f5c79368708 P 9562c35b63a44110baa2927f0aebd3ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9562c35b63a44110baa2927f0aebd3ea" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 40467 } } peers { permanent_uuid: "ca35ccb669e445c795c89d47d72466a9" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 34029 } } peers { permanent_uuid: "ae4773cc6b924efbbcefec0f1f0c0077" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 44483 } }
I20260811 19:11:04.880612 5172 raft_consensus.cc:374] T cde62da5febe4e4197881f5c79368708 P 9562c35b63a44110baa2927f0aebd3ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:11:04.880702 5172 raft_consensus.cc:729] T cde62da5febe4e4197881f5c79368708 P 9562c35b63a44110baa2927f0aebd3ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9562c35b63a44110baa2927f0aebd3ea, State: Initialized, Role: FOLLOWER
I20260811 19:11:04.881417 5172 consensus_queue.cc:260] T cde62da5febe4e4197881f5c79368708 P 9562c35b63a44110baa2927f0aebd3ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9562c35b63a44110baa2927f0aebd3ea" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 40467 } } peers { permanent_uuid: "ca35ccb669e445c795c89d47d72466a9" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 34029 } } peers { permanent_uuid: "ae4773cc6b924efbbcefec0f1f0c0077" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 44483 } }
I20260811 19:11:04.881606 5173 ts_tablet_manager.cc:1428] T cde62da5febe4e4197881f5c79368708 P ae4773cc6b924efbbcefec0f1f0c0077: Time spent starting tablet: real 0.013s user 0.012s sys 0.002s
I20260811 19:11:04.882555 5144 heartbeater.cc:503] Master 127.3.61.190:35085 was elected leader, sending a full tablet report...
I20260811 19:11:04.883208 5172 ts_tablet_manager.cc:1428] T cde62da5febe4e4197881f5c79368708 P 9562c35b63a44110baa2927f0aebd3ea: Time spent starting tablet: real 0.016s user 0.010s sys 0.003s
W20260811 19:11:04.914691 4924 tablet.cc:2369] T cde62da5febe4e4197881f5c79368708 P ca35ccb669e445c795c89d47d72466a9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 19:11:04.954571 4687 tablet.cc:2369] T cde62da5febe4e4197881f5c79368708 P ae4773cc6b924efbbcefec0f1f0c0077: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 19:11:04.962132 5145 tablet.cc:2369] T cde62da5febe4e4197881f5c79368708 P 9562c35b63a44110baa2927f0aebd3ea: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:11:05.082511 5181 raft_consensus.cc:482] T cde62da5febe4e4197881f5c79368708 P 9562c35b63a44110baa2927f0aebd3ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:11:05.082846 5181 raft_consensus.cc:504] T cde62da5febe4e4197881f5c79368708 P 9562c35b63a44110baa2927f0aebd3ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9562c35b63a44110baa2927f0aebd3ea" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 40467 } } peers { permanent_uuid: "ca35ccb669e445c795c89d47d72466a9" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 34029 } } peers { permanent_uuid: "ae4773cc6b924efbbcefec0f1f0c0077" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 44483 } }
I20260811 19:11:05.084821 5181 leader_election.cc:288] T cde62da5febe4e4197881f5c79368708 P 9562c35b63a44110baa2927f0aebd3ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca35ccb669e445c795c89d47d72466a9 (127.3.61.130:34029), ae4773cc6b924efbbcefec0f1f0c0077 (127.3.61.129:44483)
I20260811 19:11:05.091063 5148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:05.085315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:44483 (local address 127.3.61.131:46801)
0811 19:11:05.085509 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:05.085530 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:05.085690 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:05.086099 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:05.086114 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:05.086167 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:11:05.086451 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:05.086468 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:05.087514 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:05.087520 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:05.089162 (+ 1642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:05.089177 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:05.090418 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:05.090424 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:05.090610 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:05.090638 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 19:11:05.090731 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 19:11:05.090833 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260811 19:11:05.091210 5168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:05.085370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:46801 (local address 127.3.61.129:44483)
0811 19:11:05.085592 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:05.085597 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:05.085619 (+ 22us) server_negotiation.cc:407] Connection header received
0811 19:11:05.085850 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:05.085856 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:05.085938 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:05.086035 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:11:05.086637 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:05.087379 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:05.089367 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:05.090254 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:05.090390 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:05.090445 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:05.090804 (+ 359us) server_negotiation.cc:299] Negotiation successful
0811 19:11:05.090962 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 19:11:05.092760 4607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cde62da5febe4e4197881f5c79368708" candidate_uuid: "9562c35b63a44110baa2927f0aebd3ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae4773cc6b924efbbcefec0f1f0c0077" is_pre_election: true
I20260811 19:11:05.093281 5185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:05.085340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.130:34029 (local address 127.3.61.131:34501)
0811 19:11:05.086657 (+ 1317us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:05.086674 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:05.086910 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:05.087242 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:05.087248 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:05.087279 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:11:05.087735 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:05.087748 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:05.088808 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:05.088815 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:05.090396 (+ 1581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:05.090410 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:05.092332 (+ 1922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:05.092339 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:05.092517 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:05.092553 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 19:11:05.092693 (+ 140us) client_negotiation.cc:240] Negotiation successful
0811 19:11:05.092856 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1186,"mutex_wait_us":444,"thread_start_us":470,"threads_started":1}
I20260811 19:11:05.093379 5164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:05.085337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:34501 (local address 127.3.61.130:34029)
0811 19:11:05.085635 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:05.085640 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:05.086864 (+ 1224us) server_negotiation.cc:407] Connection header received
0811 19:11:05.087050 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:05.087055 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:05.087120 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:05.087205 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:11:05.087952 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:05.088640 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:05.090758 (+ 2118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:05.092000 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:05.092125 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:05.092188 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:05.092828 (+ 640us) server_negotiation.cc:299] Negotiation successful
0811 19:11:05.093061 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135}
I20260811 19:11:05.093513 4607 raft_consensus.cc:2454] T cde62da5febe4e4197881f5c79368708 P ae4773cc6b924efbbcefec0f1f0c0077 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9562c35b63a44110baa2927f0aebd3ea in term 0.
I20260811 19:11:05.094358 4956 leader_election.cc:302] T cde62da5febe4e4197881f5c79368708 P 9562c35b63a44110baa2927f0aebd3ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9562c35b63a44110baa2927f0aebd3ea, ae4773cc6b924efbbcefec0f1f0c0077; no voters:
I20260811 19:11:05.094447 4842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cde62da5febe4e4197881f5c79368708" candidate_uuid: "9562c35b63a44110baa2927f0aebd3ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca35ccb669e445c795c89d47d72466a9" is_pre_election: true
I20260811 19:11:05.094934 4842 raft_consensus.cc:2454] T cde62da5febe4e4197881f5c79368708 P ca35ccb669e445c795c89d47d72466a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9562c35b63a44110baa2927f0aebd3ea in term 0.
I20260811 19:11:05.095026 5181 raft_consensus.cc:2781] T cde62da5febe4e4197881f5c79368708 P 9562c35b63a44110baa2927f0aebd3ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:11:05.095263 5181 raft_consensus.cc:482] T cde62da5febe4e4197881f5c79368708 P 9562c35b63a44110baa2927f0aebd3ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:11:05.095395 5181 raft_consensus.cc:3037] T cde62da5febe4e4197881f5c79368708 P 9562c35b63a44110baa2927f0aebd3ea [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:11:05.097802 5181 raft_consensus.cc:504] T cde62da5febe4e4197881f5c79368708 P 9562c35b63a44110baa2927f0aebd3ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9562c35b63a44110baa2927f0aebd3ea" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 40467 } } peers { permanent_uuid: "ca35ccb669e445c795c89d47d72466a9" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 34029 } } peers { permanent_uuid: "ae4773cc6b924efbbcefec0f1f0c0077" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 44483 } }
I20260811 19:11:05.098631 5181 leader_election.cc:288] T cde62da5febe4e4197881f5c79368708 P 9562c35b63a44110baa2927f0aebd3ea [CANDIDATE]: Term 1 election: Requested vote from peers ca35ccb669e445c795c89d47d72466a9 (127.3.61.130:34029), ae4773cc6b924efbbcefec0f1f0c0077 (127.3.61.129:44483)
I20260811 19:11:05.099126 4842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cde62da5febe4e4197881f5c79368708" candidate_uuid: "9562c35b63a44110baa2927f0aebd3ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca35ccb669e445c795c89d47d72466a9"
I20260811 19:11:05.099205 4607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cde62da5febe4e4197881f5c79368708" candidate_uuid: "9562c35b63a44110baa2927f0aebd3ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae4773cc6b924efbbcefec0f1f0c0077"
I20260811 19:11:05.099371 4842 raft_consensus.cc:3037] T cde62da5febe4e4197881f5c79368708 P ca35ccb669e445c795c89d47d72466a9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:11:05.099409 4607 raft_consensus.cc:3037] T cde62da5febe4e4197881f5c79368708 P ae4773cc6b924efbbcefec0f1f0c0077 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:11:05.102766 4607 raft_consensus.cc:2454] T cde62da5febe4e4197881f5c79368708 P ae4773cc6b924efbbcefec0f1f0c0077 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9562c35b63a44110baa2927f0aebd3ea in term 1.
I20260811 19:11:05.102766 4842 raft_consensus.cc:2454] T cde62da5febe4e4197881f5c79368708 P ca35ccb669e445c795c89d47d72466a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9562c35b63a44110baa2927f0aebd3ea in term 1.
I20260811 19:11:05.103361 4954 leader_election.cc:302] T cde62da5febe4e4197881f5c79368708 P 9562c35b63a44110baa2927f0aebd3ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9562c35b63a44110baa2927f0aebd3ea, ca35ccb669e445c795c89d47d72466a9; no voters:
I20260811 19:11:05.103816 5181 raft_consensus.cc:2781] T cde62da5febe4e4197881f5c79368708 P 9562c35b63a44110baa2927f0aebd3ea [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:11:05.104386 5181 raft_consensus.cc:686] T cde62da5febe4e4197881f5c79368708 P 9562c35b63a44110baa2927f0aebd3ea [term 1 LEADER]: Becoming Leader. State: Replica: 9562c35b63a44110baa2927f0aebd3ea, State: Running, Role: LEADER
I20260811 19:11:05.104983 5181 consensus_queue.cc:237] T cde62da5febe4e4197881f5c79368708 P 9562c35b63a44110baa2927f0aebd3ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9562c35b63a44110baa2927f0aebd3ea" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 40467 } } peers { permanent_uuid: "ca35ccb669e445c795c89d47d72466a9" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 34029 } } peers { permanent_uuid: "ae4773cc6b924efbbcefec0f1f0c0077" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 44483 } }
I20260811 19:11:05.110124 4341 catalog_manager.cc:5495] T cde62da5febe4e4197881f5c79368708 P 9562c35b63a44110baa2927f0aebd3ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 9562c35b63a44110baa2927f0aebd3ea (127.3.61.131). New cstate: current_term: 1 leader_uuid: "9562c35b63a44110baa2927f0aebd3ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9562c35b63a44110baa2927f0aebd3ea" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 40467 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca35ccb669e445c795c89d47d72466a9" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 34029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ae4773cc6b924efbbcefec0f1f0c0077" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 44483 } health_report { overall_health: UNKNOWN } } }
Aug 11 19:11:05 dist-test-slave-t7g5 krb5kdc[4265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475462, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.129@KRBTEST.COM
I20260811 19:11:05.139245 5168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:05.118767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52134 (local address 127.3.61.129:44483)
0811 19:11:05.119048 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:05.119055 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:05.119400 (+ 345us) server_negotiation.cc:407] Connection header received
0811 19:11:05.119610 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:05.119614 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:05.119692 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:05.119801 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:05.122052 (+ 2251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:05.123006 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:05.124595 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:05.125152 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:05.125308 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:05.132925 (+ 7617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:05.132949 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:05.132969 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:05.133011 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:05.135992 (+ 2981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:05.136938 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:05.136946 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:05.136949 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:05.137090 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:05.137591 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:05.137595 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:05.137597 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:05.138039 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:05.138297 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:05.138655 (+ 358us) server_negotiation.cc:299] Negotiation successful
0811 19:11:05.138872 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
Aug 11 19:11:05 dist-test-slave-t7g5 krb5kdc[4265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475465, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 19:11:05 dist-test-slave-t7g5 krb5kdc[4265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475465, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.3.61.129@KRBTEST.COM
I20260811 19:11:05.199560 5168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:05.184404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52140 (local address 127.3.61.129:44483)
0811 19:11:05.184725 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:05.184734 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:05.185153 (+ 419us) server_negotiation.cc:407] Connection header received
0811 19:11:05.185349 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:05.185353 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:05.185429 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:05.185515 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:05.187023 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:05.187776 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:05.189101 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:05.189527 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:05.189688 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:05.193929 (+ 4241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:05.193935 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:05.193939 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:05.193976 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:05.196744 (+ 2768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:05.197427 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:05.197432 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:05.197434 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:05.197582 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:05.198119 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:05.198123 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:05.198125 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:05.198437 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:05.198688 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:05.198994 (+ 306us) server_negotiation.cc:299] Negotiation successful
0811 19:11:05.199223 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133}
Aug 11 19:11:05 dist-test-slave-t7g5 krb5kdc[4265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475465, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 19:11:05 dist-test-slave-t7g5 krb5kdc[4265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475465, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.129@KRBTEST.COM
I20260811 19:11:05.256634 5168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:05.242119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52152 (local address 127.3.61.129:44483)
0811 19:11:05.242345 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:05.242351 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:05.242672 (+ 321us) server_negotiation.cc:407] Connection header received
0811 19:11:05.242859 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:05.242864 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:05.242931 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:05.243015 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:05.244446 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:05.245140 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:05.246450 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:05.246881 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:05.246994 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:05.251002 (+ 4008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:05.251007 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:05.251010 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:05.251059 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:05.253924 (+ 2865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:05.254639 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:05.254643 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:05.254645 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:05.254769 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:05.255125 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:05.255129 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:05.255132 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:05.255480 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:05.255757 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:05.256051 (+ 294us) server_negotiation.cc:299] Negotiation successful
0811 19:11:05.256296 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260811 19:11:05.258759 4529 tablet_service.cc:2908] Rejecting scan request for tablet cde62da5febe4e4197881f5c79368708: Uninitialized: clean time has not yet been initialized
I20260811 19:11:05.262223 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 4452
I20260811 19:11:05.277395 5168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:05.263261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52156 (local address 127.3.61.129:44483)
0811 19:11:05.263433 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:05.263440 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:05.263913 (+ 473us) server_negotiation.cc:407] Connection header received
0811 19:11:05.264033 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:05.264037 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:05.264116 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:05.264202 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:05.265418 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:05.266171 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:05.267494 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:05.267934 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:05.268048 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:05.271078 (+ 3030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:05.271084 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:05.271088 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:05.271127 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:05.274196 (+ 3069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:05.275265 (+ 1069us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:05.275271 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:05.275273 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:05.275445 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:05.275951 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:05.275956 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:05.275959 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:05.276292 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:05.276666 (+ 374us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:05.276963 (+ 297us) server_negotiation.cc:299] Negotiation successful
0811 19:11:05.277149 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 19:11:05.278226 4677 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:11:05.371625 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 4452
I20260811 19:11:05.393638 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 4693
Aug 11 19:11:05 dist-test-slave-t7g5 krb5kdc[4265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475465, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.130@KRBTEST.COM
I20260811 19:11:05.410961 5164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:05.394811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41140 (local address 127.3.61.130:34029)
0811 19:11:05.395004 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:05.395011 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:05.395035 (+ 24us) server_negotiation.cc:407] Connection header received
0811 19:11:05.395253 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:05.395259 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:05.395346 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:05.395463 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:05.396991 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:05.397969 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:05.399332 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:05.399757 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:05.399874 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:05.404820 (+ 4946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:05.404838 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:05.404857 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:05.404904 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:05.407900 (+ 2996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:05.408689 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:05.408697 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:05.408699 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:05.408856 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:05.409360 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:05.409365 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:05.409367 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:05.409895 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:05.410163 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:05.410433 (+ 270us) server_negotiation.cc:299] Negotiation successful
0811 19:11:05.410628 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 19:11:05.411702 4913 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:11:05.507840 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 4693
I20260811 19:11:05.534052 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 4933
Aug 11 19:11:05 dist-test-slave-t7g5 krb5kdc[4265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475465, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.131@KRBTEST.COM
I20260811 19:11:05.551662 5213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:05.534910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44624 (local address 127.3.61.131:40467)
0811 19:11:05.535412 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:05.535417 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:05.535441 (+ 24us) server_negotiation.cc:407] Connection header received
0811 19:11:05.535519 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:05.535523 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:05.535693 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:05.535812 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:05.537243 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:05.538159 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:05.539994 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:05.540437 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:05.540553 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:05.545078 (+ 4525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:05.545095 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:05.545114 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:05.545152 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:05.548092 (+ 2940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:05.548919 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:05.548928 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:05.548931 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:05.549116 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:05.549571 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:05.549576 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:05.549578 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:05.550067 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:05.550332 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:05.550828 (+ 496us) server_negotiation.cc:299] Negotiation successful
0811 19:11:05.551322 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":264,"threads_started":1}
I20260811 19:11:05.552392 5138 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:11:05.658285 5148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:05.656653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:44483 (local address 127.3.61.131:35219)
0811 19:11:05.657813 (+ 1160us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:05.658036 (+ 223us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.129:44483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":905}
W20260811 19:11:05.658957 4956 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.61.129:44483: connect: Connection refused (error 111)
I20260811 19:11:05.659159 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 4933
I20260811 19:11:05.690958 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 4290
I20260811 19:11:05.692298 4391 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:11:05.808619 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 4290
2026-08-11T19:11:05Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (5246 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:11:05 dist-test-slave-t7g5 krb5kdc[5220](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:11:05 dist-test-slave-t7g5 krb5kdc[5220](info): set up 2 sockets
Aug 11 19:11:05 dist-test-slave-t7g5 krb5kdc[5220](info): commencing operation
krb5kdc: starting...
W20260811 19:11:07.121447 3318 mini_kdc.cc:121] Time spent starting KDC: real 1.257s user 0.000s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Aug 11 19:11:07 dist-test-slave-t7g5 krb5kdc[5220](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:11:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:11:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.61.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab.
Entry for principal kudu/127.3.61.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab.
I20260811 19:11:07.382951 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:42953
--webserver_interface=127.3.61.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:38581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.61.190:42953
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:11:07.766938 5242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:11:07.767251 5242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:11:07.767407 5242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:11:07.776389 5242 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:11:07.776659 5242 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 19:11:07.776746 5242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:11:07.776844 5242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:11:07.776918 5242 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:11:07.777024 5242 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:11:07.788621 5242 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:38581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.61.190:42953
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:42953
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.61.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:11:07.790045 5242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:11:07.791458 5242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:11:07.806420 5250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:07.806439 5251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:07.806640 5255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:11:07.807255 5242 server_base.cc:1034] running on GCE node
I20260811 19:11:07.808173 5242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:11:07.809787 5242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:11:07.811045 5242 hybrid_clock.cc:648] HybridClock initialized: now 1786475467810964 us; error 57 us; skew 500 ppm
I20260811 19:11:07.811414 5242 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:11:07 dist-test-slave-t7g5 krb5kdc[5220](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475467, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:11:07.815929 5242 init.cc:377] Logged in from keytab as kudu/127.3.61.190@KRBTEST.COM (short username kudu)
I20260811 19:11:07.818256 5242 webserver.cc:460] Webserver started at http://127.3.61.190:39115/ using document root <none> and password file <none>
I20260811 19:11:07.819082 5242 fs_manager.cc:359] Metadata directory not provided
I20260811 19:11:07.819186 5242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:11:07.819670 5242 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:11:07.823238 5242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/master-0/data/instance:
uuid: "6d3b329daf2c495a81783be32467ea40"
format_stamp: "Formatted at 2026-08-11 19:11:07 on dist-test-slave-t7g5"
server_key: "8fff344f1518653c012eed56aeba6bfa"
server_key_iv: "34ef06ef27dc1157a72e22f37af55f0a"
server_key_version: "encryptionkey@0"
I20260811 19:11:07.824110 5242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/master-0/wal/instance:
uuid: "6d3b329daf2c495a81783be32467ea40"
format_stamp: "Formatted at 2026-08-11 19:11:07 on dist-test-slave-t7g5"
server_key: "8fff344f1518653c012eed56aeba6bfa"
server_key_iv: "34ef06ef27dc1157a72e22f37af55f0a"
server_key_version: "encryptionkey@0"
I20260811 19:11:07.829738 5242 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 19:11:07.833834 5267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:07.836308 5242 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260811 19:11:07.836536 5242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/master-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/master-0/wal
uuid: "6d3b329daf2c495a81783be32467ea40"
format_stamp: "Formatted at 2026-08-11 19:11:07 on dist-test-slave-t7g5"
server_key: "8fff344f1518653c012eed56aeba6bfa"
server_key_iv: "34ef06ef27dc1157a72e22f37af55f0a"
server_key_version: "encryptionkey@0"
I20260811 19:11:07.836781 5242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:11:07.863646 5242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:11:07.868304 5242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:11:07.868680 5242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:11:07.894946 5242 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.190:42953
I20260811 19:11:07.894949 5383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.190:42953 every 8 connection(s)
I20260811 19:11:07.897169 5242 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
I20260811 19:11:07.900120 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 5242
I20260811 19:11:07.900384 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/master-0/wal/instance
I20260811 19:11:07.901239 3318 external_mini_cluster.cc:1428] Setting key a5d51e653f324f162b04c77c849041d0
I20260811 19:11:07.903214 5384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 19:11:07 dist-test-slave-t7g5 krb5kdc[5220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:11:07.921072 5384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d3b329daf2c495a81783be32467ea40: Bootstrap starting.
I20260811 19:11:07.924890 5384 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d3b329daf2c495a81783be32467ea40: Neither blocks nor log segments found. Creating new log.
I20260811 19:11:07.926162 5384 log.cc:824] T 00000000000000000000000000000000 P 6d3b329daf2c495a81783be32467ea40: Log is configured to *not* fsync() on all Append() calls
I20260811 19:11:07.929768 5384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d3b329daf2c495a81783be32467ea40: No bootstrap required, opened a new log
I20260811 19:11:07.930821 5387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:07.903581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53494 (local address 127.3.61.190:42953)
0811 19:11:07.904772 (+ 1191us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:07.904784 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:07.904837 (+ 53us) server_negotiation.cc:407] Connection header received
0811 19:11:07.906551 (+ 1714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:07.906574 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:07.906997 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:07.907341 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:07.909214 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:07.910878 (+ 1664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:07.912912 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:07.916063 (+ 3151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:07.916329 (+ 266us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:07.922303 (+ 5974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:07.922314 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:07.922334 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:07.922390 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:07.925598 (+ 3208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:07.926630 (+ 1032us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:07.926638 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:07.926649 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:07.926940 (+ 291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:07.927506 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:07.927517 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:07.927520 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:07.928183 (+ 663us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:07.928508 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:07.928999 (+ 491us) server_negotiation.cc:299] Negotiation successful
0811 19:11:07.929528 (+ 529us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":847,"thread_start_us":612,"threads_started":1}
I20260811 19:11:07.937855 5384 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d3b329daf2c495a81783be32467ea40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d3b329daf2c495a81783be32467ea40" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 42953 } }
I20260811 19:11:07.938272 5384 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d3b329daf2c495a81783be32467ea40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:11:07.938448 5384 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d3b329daf2c495a81783be32467ea40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d3b329daf2c495a81783be32467ea40, State: Initialized, Role: FOLLOWER
I20260811 19:11:07.939522 5384 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d3b329daf2c495a81783be32467ea40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d3b329daf2c495a81783be32467ea40" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 42953 } }
I20260811 19:11:07.939935 5384 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d3b329daf2c495a81783be32467ea40 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:11:07.940099 5384 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d3b329daf2c495a81783be32467ea40 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:11:07.940304 5384 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d3b329daf2c495a81783be32467ea40 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:11:07.943259 5384 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d3b329daf2c495a81783be32467ea40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d3b329daf2c495a81783be32467ea40" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 42953 } }
I20260811 19:11:07.943913 5384 leader_election.cc:302] T 00000000000000000000000000000000 P 6d3b329daf2c495a81783be32467ea40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d3b329daf2c495a81783be32467ea40; no voters:
I20260811 19:11:07.944525 5384 leader_election.cc:288] T 00000000000000000000000000000000 P 6d3b329daf2c495a81783be32467ea40 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:11:07.944635 5389 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d3b329daf2c495a81783be32467ea40 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:11:07.945372 5389 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d3b329daf2c495a81783be32467ea40 [term 1 LEADER]: Becoming Leader. State: Replica: 6d3b329daf2c495a81783be32467ea40, State: Running, Role: LEADER
I20260811 19:11:07.946053 5389 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d3b329daf2c495a81783be32467ea40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d3b329daf2c495a81783be32467ea40" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 42953 } }
I20260811 19:11:07.946605 5384 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d3b329daf2c495a81783be32467ea40 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:11:07.948745 5395 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d3b329daf2c495a81783be32467ea40 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d3b329daf2c495a81783be32467ea40. Latest consensus state: current_term: 1 leader_uuid: "6d3b329daf2c495a81783be32467ea40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d3b329daf2c495a81783be32467ea40" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 42953 } } }
I20260811 19:11:07.949327 5395 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d3b329daf2c495a81783be32467ea40 [sys.catalog]: This master's current role is: LEADER
I20260811 19:11:07.948931 5390 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d3b329daf2c495a81783be32467ea40 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6d3b329daf2c495a81783be32467ea40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d3b329daf2c495a81783be32467ea40" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 42953 } } }
I20260811 19:11:07.949537 5390 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d3b329daf2c495a81783be32467ea40 [sys.catalog]: This master's current role is: LEADER
I20260811 19:11:07.950279 5400 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:11:07.959978 5400 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:11:07.971158 5400 catalog_manager.cc:1347] Generated new cluster ID: 25a6a44497f44903a7a745b28281ebc9
I20260811 19:11:07.971304 5400 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:11:07.987821 5400 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:11:07.989929 5400 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:11:08.002847 5400 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d3b329daf2c495a81783be32467ea40: Generated new TSK 0
I20260811 19:11:08.003926 5400 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.61.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab.
Entry for principal kudu/127.3.61.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab.
I20260811 19:11:08.069912 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.129:0
--local_ip_for_outbound_sockets=127.3.61.129
--webserver_interface=127.3.61.129
--webserver_port=0
--tserver_master_addrs=127.3.61.190:42953
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab
--principal=kudu/127.3.61.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:38581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:11:08.474078 5428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:11:08.474424 5428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:11:08.474558 5428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:11:08.482020 5428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:11:08.482172 5428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:11:08.482300 5428 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 19:11:08.482424 5428 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.129
I20260811 19:11:08.492961 5428 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:38581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab
--principal=kudu/127.3.61.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.61.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.61.190:42953
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.129
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:11:08.494405 5428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:11:08.495945 5428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:11:08.510208 5437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:08.510268 5440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:08.510237 5444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:11:08.511468 5428 server_base.cc:1034] running on GCE node
I20260811 19:11:08.512182 5428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:11:08.513250 5428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:11:08.514644 5428 hybrid_clock.cc:648] HybridClock initialized: now 1786475468514505 us; error 36 us; skew 500 ppm
I20260811 19:11:08.515182 5428 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:11:08 dist-test-slave-t7g5 krb5kdc[5220](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475468, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:11:08.519353 5428 init.cc:377] Logged in from keytab as kudu/127.3.61.129@KRBTEST.COM (short username kudu)
I20260811 19:11:08.521754 5428 webserver.cc:460] Webserver started at http://127.3.61.129:37913/ using document root <none> and password file <none>
I20260811 19:11:08.522778 5428 fs_manager.cc:359] Metadata directory not provided
I20260811 19:11:08.522919 5428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:11:08.523291 5428 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:11:08.526715 5428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/ts-0/data/instance:
uuid: "02c4a487189f44b69b7f49f33a486807"
format_stamp: "Formatted at 2026-08-11 19:11:08 on dist-test-slave-t7g5"
server_key: "a5c90e15d3fdb11a917d1e3397c9a672"
server_key_iv: "d5b453ba838753db43d65efd6d08cb74"
server_key_version: "encryptionkey@0"
I20260811 19:11:08.527592 5428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/ts-0/wal/instance:
uuid: "02c4a487189f44b69b7f49f33a486807"
format_stamp: "Formatted at 2026-08-11 19:11:08 on dist-test-slave-t7g5"
server_key: "a5c90e15d3fdb11a917d1e3397c9a672"
server_key_iv: "d5b453ba838753db43d65efd6d08cb74"
server_key_version: "encryptionkey@0"
I20260811 19:11:08.534029 5428 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 19:11:08.539039 5455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:08.540860 5428 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260811 19:11:08.541083 5428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/ts-0/wal
uuid: "02c4a487189f44b69b7f49f33a486807"
format_stamp: "Formatted at 2026-08-11 19:11:08 on dist-test-slave-t7g5"
server_key: "a5c90e15d3fdb11a917d1e3397c9a672"
server_key_iv: "d5b453ba838753db43d65efd6d08cb74"
server_key_version: "encryptionkey@0"
I20260811 19:11:08.541335 5428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:11:08.566926 5428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:11:08.572042 5428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:11:08.572304 5428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:11:08.576776 5428 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:11:08.576952 5428 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:08.577248 5428 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:11:08.577548 5428 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Aug 11 19:11:08 dist-test-slave-t7g5 krb5kdc[5220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475468, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.129@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:11:08.603341 5470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:08.577512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:39141 (local address 127.3.61.190:42953)
0811 19:11:08.578074 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:08.578081 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:08.579741 (+ 1660us) server_negotiation.cc:407] Connection header received
0811 19:11:08.582125 (+ 2384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:08.582132 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:08.582221 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:08.582322 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:08.584447 (+ 2125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:08.585592 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:08.588800 (+ 3208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:08.589215 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:08.589342 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:08.594846 (+ 5504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:08.594854 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:08.594859 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:08.594906 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:08.598276 (+ 3370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:08.599252 (+ 976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:08.599258 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:08.599262 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:08.599411 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:08.600501 (+ 1090us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:08.600507 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:08.600511 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:08.600813 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:08.600938 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:08.602727 (+ 1789us) server_negotiation.cc:299] Negotiation successful
0811 19:11:08.603004 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":311,"threads_started":1}
I20260811 19:11:08.605937 5474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:08.578550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:42953 (local address 127.3.61.129:39141)
0811 19:11:08.579524 (+ 974us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:08.579577 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:08.581784 (+ 2207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:08.582623 (+ 839us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:08.582632 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:08.583238 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:11:08.584137 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:08.584162 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:08.585888 (+ 1726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:08.585894 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:08.588606 (+ 2712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:08.588628 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:08.589507 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:08.589515 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:08.589825 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:08.590635 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:11:08.590676 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:11:08.594542 (+ 3866us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:11:08.598437 (+ 3895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:08.598446 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:08.598473 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:08.599124 (+ 651us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:08.599655 (+ 531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:08.599663 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:08.599672 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:08.600291 (+ 619us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:08.601012 (+ 721us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:11:08.601024 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:11:08.601921 (+ 897us) client_negotiation.cc:769] Sending connection context
0811 19:11:08.602497 (+ 576us) client_negotiation.cc:240] Negotiation successful
0811 19:11:08.603478 (+ 981us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":695,"thread_start_us":476,"threads_started":1}
I20260811 19:11:08.623471 5428 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.129:42915
I20260811 19:11:08.623505 5700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.129:42915 every 8 connection(s)
I20260811 19:11:08.624887 5428 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
I20260811 19:11:08.627604 5704 heartbeater.cc:347] Connected to a master server at 127.3.61.190:42953
I20260811 19:11:08.627926 5704 heartbeater.cc:464] Registering TS with master...
I20260811 19:11:08.628866 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 5428
I20260811 19:11:08.628969 5704 heartbeater.cc:511] Master 127.3.61.190:42953 requested a full tablet report, sending...
I20260811 19:11:08.629122 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/ts-0/wal/instance
I20260811 19:11:08.630033 3318 external_mini_cluster.cc:1428] Setting key 8fe3243ff9d79b30bb573419bde38c58
I20260811 19:11:08.632025 5300 ts_manager.cc:194] Registered new tserver with Master: 02c4a487189f44b69b7f49f33a486807 (127.3.61.129:42915)
I20260811 19:11:08.634267 5300 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.129@KRBTEST.COM'} at 127.3.61.129:39141
I20260811 19:11:08.645237 5470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:08.617209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:35179 (local address 127.3.61.190:42953)
0811 19:11:08.617504 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:08.617511 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:08.632257 (+ 14746us) server_negotiation.cc:407] Connection header received
0811 19:11:08.632747 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:08.632753 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:08.632846 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:08.632923 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:08.634248 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:08.635438 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:08.637009 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:08.637486 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:08.637605 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:08.639787 (+ 2182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:08.639792 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:08.639796 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:08.639830 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:08.642479 (+ 2649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:08.643218 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:08.643223 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:08.643225 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:08.643366 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:08.643731 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:08.643735 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:08.643737 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:08.644088 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:08.644224 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:08.644643 (+ 419us) server_negotiation.cc:299] Negotiation successful
0811 19:11:08.644867 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":58,"server-negotiator.queue_time_us":119}
I20260811 19:11:08.645305 5675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:08.617171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:42953 (local address 127.3.61.129:35179)
0811 19:11:08.632128 (+ 14957us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:08.632158 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:08.632589 (+ 431us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:08.633063 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:08.633068 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:08.633698 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:11:08.634031 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:08.634045 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:08.635705 (+ 1660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:08.635709 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:08.636835 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:08.636859 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:08.637723 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:08.637730 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:08.637892 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:08.638755 (+ 863us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:11:08.638775 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:11:08.639563 (+ 788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:11:08.642620 (+ 3057us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:08.642626 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:08.642630 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:08.643084 (+ 454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:08.643480 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:08.643485 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:08.643489 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:08.643620 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:08.644209 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:11:08.644213 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:11:08.644383 (+ 170us) client_negotiation.cc:769] Sending connection context
0811 19:11:08.644504 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 19:11:08.644818 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14737,"mutex_wait_us":2,"thread_start_us":15098,"threads_started":1}
WARNING: no policy specified for kudu/127.3.61.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab.
Entry for principal kudu/127.3.61.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab.
I20260811 19:11:08.689347 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.130:0
--local_ip_for_outbound_sockets=127.3.61.130
--webserver_interface=127.3.61.130
--webserver_port=0
--tserver_master_addrs=127.3.61.190:42953
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab
--principal=kudu/127.3.61.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:38581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:11:09.058408 5708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:11:09.058761 5708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:11:09.058908 5708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:11:09.066486 5708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:11:09.066649 5708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:11:09.066776 5708 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 19:11:09.066893 5708 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.130
I20260811 19:11:09.077420 5708 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:38581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab
--principal=kudu/127.3.61.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.61.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.61.190:42953
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.130
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:11:09.078876 5708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:11:09.080320 5708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:11:09.094477 5718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:09.094497 5716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:09.095155 5720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:11:09.095726 5708 server_base.cc:1034] running on GCE node
I20260811 19:11:09.096524 5708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:11:09.097741 5708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:11:09.099007 5708 hybrid_clock.cc:648] HybridClock initialized: now 1786475469098924 us; error 58 us; skew 500 ppm
I20260811 19:11:09.099345 5708 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:11:09 dist-test-slave-t7g5 krb5kdc[5220](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475469, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:11:09.102954 5708 init.cc:377] Logged in from keytab as kudu/127.3.61.130@KRBTEST.COM (short username kudu)
I20260811 19:11:09.104781 5708 webserver.cc:460] Webserver started at http://127.3.61.130:39623/ using document root <none> and password file <none>
I20260811 19:11:09.105638 5708 fs_manager.cc:359] Metadata directory not provided
I20260811 19:11:09.105799 5708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:11:09.106139 5708 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:11:09.109318 5708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/ts-1/data/instance:
uuid: "df6626ebdf524b78a3551525563ac21d"
format_stamp: "Formatted at 2026-08-11 19:11:09 on dist-test-slave-t7g5"
server_key: "a19b0ac267541fae5b1ba4ff593b1a29"
server_key_iv: "edab250ee5051f11b055682a0499914a"
server_key_version: "encryptionkey@0"
I20260811 19:11:09.110258 5708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/ts-1/wal/instance:
uuid: "df6626ebdf524b78a3551525563ac21d"
format_stamp: "Formatted at 2026-08-11 19:11:09 on dist-test-slave-t7g5"
server_key: "a19b0ac267541fae5b1ba4ff593b1a29"
server_key_iv: "edab250ee5051f11b055682a0499914a"
server_key_version: "encryptionkey@0"
I20260811 19:11:09.116535 5708 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 19:11:09.121258 5738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:09.123431 5708 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 19:11:09.123653 5708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/ts-1/wal
uuid: "df6626ebdf524b78a3551525563ac21d"
format_stamp: "Formatted at 2026-08-11 19:11:09 on dist-test-slave-t7g5"
server_key: "a19b0ac267541fae5b1ba4ff593b1a29"
server_key_iv: "edab250ee5051f11b055682a0499914a"
server_key_version: "encryptionkey@0"
I20260811 19:11:09.123898 5708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:11:09.136274 5708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:11:09.140762 5708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:11:09.141172 5708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:11:09.144323 5708 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:11:09.144443 5708 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:09.144632 5708 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:11:09.144771 5708 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Aug 11 19:11:09 dist-test-slave-t7g5 krb5kdc[5220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475469, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.130@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:11:09.167915 5470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:09.144844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:35127 (local address 127.3.61.190:42953)
0811 19:11:09.145020 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:09.145027 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:09.146901 (+ 1874us) server_negotiation.cc:407] Connection header received
0811 19:11:09.149072 (+ 2171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:09.149078 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:09.149171 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:09.149261 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:09.151074 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:09.151814 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:09.153467 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:09.154155 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:09.154293 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:09.160332 (+ 6039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:09.160338 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:09.160342 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:09.160396 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:09.163865 (+ 3469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:09.165065 (+ 1200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:09.165071 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:09.165074 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:09.165231 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:09.165984 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:09.165990 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:09.165994 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:09.166307 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:09.166479 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:09.167300 (+ 821us) server_negotiation.cc:299] Negotiation successful
0811 19:11:09.167555 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 19:11:09.169140 5763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:09.145589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:42953 (local address 127.3.61.130:35127)
0811 19:11:09.146727 (+ 1138us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:09.146776 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:09.148785 (+ 2009us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:09.149603 (+ 818us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:09.149617 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:09.150291 (+ 674us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:11:09.150894 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:09.150913 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:09.151968 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:09.151973 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:09.153297 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:09.153311 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:09.154475 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:09.154484 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:09.154894 (+ 410us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:09.156283 (+ 1389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:11:09.156322 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:11:09.160037 (+ 3715us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:11:09.164085 (+ 4048us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:09.164097 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:09.164113 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:09.164863 (+ 750us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:09.165347 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:09.165352 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:09.165356 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:09.165829 (+ 473us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:09.166465 (+ 636us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:11:09.166474 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:11:09.166887 (+ 413us) client_negotiation.cc:769] Sending connection context
0811 19:11:09.167149 (+ 262us) client_negotiation.cc:240] Negotiation successful
0811 19:11:09.167774 (+ 625us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":795,"thread_start_us":487,"threads_started":1}
I20260811 19:11:09.193537 5470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:09.176270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:41903 (local address 127.3.61.190:42953)
0811 19:11:09.176658 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:09.176665 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:09.178081 (+ 1416us) server_negotiation.cc:407] Connection header received
0811 19:11:09.178323 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:09.178328 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:09.178412 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:09.178520 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:09.180162 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:09.181021 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:09.183184 (+ 2163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:09.183790 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:09.183948 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:09.186836 (+ 2888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:09.186842 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:09.186845 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:09.186880 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:09.190028 (+ 3148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:09.190794 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:09.190799 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:09.190801 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:09.190935 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:09.191423 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:09.191429 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:09.191432 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:09.191900 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:09.192050 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:09.192858 (+ 808us) server_negotiation.cc:299] Negotiation successful
0811 19:11:09.193140 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260811 19:11:09.194130 5880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:09.176190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:42953 (local address 127.3.61.130:41903)
0811 19:11:09.177927 (+ 1737us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:09.177951 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:09.178145 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:09.178590 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:09.178595 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:09.179415 (+ 820us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:11:09.179851 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:09.179869 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:09.181195 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:09.181202 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:09.182886 (+ 1684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:09.182919 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:09.184036 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:09.184045 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:09.184288 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:09.185154 (+ 866us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:11:09.185184 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:11:09.186617 (+ 1433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:11:09.190174 (+ 3557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:09.190180 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:09.190185 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:09.190660 (+ 475us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:09.191062 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:09.191067 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:09.191071 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:09.191259 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:09.192009 (+ 750us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:11:09.192013 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:11:09.192268 (+ 255us) client_negotiation.cc:769] Sending connection context
0811 19:11:09.192660 (+ 392us) client_negotiation.cc:240] Negotiation successful
0811 19:11:09.193635 (+ 975us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1543,"thread_start_us":1214,"threads_started":1}
I20260811 19:11:09.199910 5708 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.130:41691
I20260811 19:11:09.199997 5991 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.130:41691 every 8 connection(s)
I20260811 19:11:09.201469 5708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
I20260811 19:11:09.203809 5994 heartbeater.cc:347] Connected to a master server at 127.3.61.190:42953
I20260811 19:11:09.204048 5994 heartbeater.cc:464] Registering TS with master...
I20260811 19:11:09.204732 5994 heartbeater.cc:511] Master 127.3.61.190:42953 requested a full tablet report, sending...
I20260811 19:11:09.206354 5300 ts_manager.cc:194] Registered new tserver with Master: df6626ebdf524b78a3551525563ac21d (127.3.61.130:41691)
I20260811 19:11:09.207198 5300 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.130@KRBTEST.COM'} at 127.3.61.130:35127
I20260811 19:11:09.207615 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 5708
I20260811 19:11:09.207803 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/ts-1/wal/instance
I20260811 19:11:09.208251 3318 external_mini_cluster.cc:1428] Setting key 8bb120e84d7e358471318ed573113003
WARNING: no policy specified for kudu/127.3.61.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab.
Entry for principal kudu/127.3.61.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab.
I20260811 19:11:09.261554 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.131:0
--local_ip_for_outbound_sockets=127.3.61.131
--webserver_interface=127.3.61.131
--webserver_port=0
--tserver_master_addrs=127.3.61.190:42953
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab
--principal=kudu/127.3.61.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:38581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:11:09.638631 5704 heartbeater.cc:503] Master 127.3.61.190:42953 was elected leader, sending a full tablet report...
W20260811 19:11:09.642894 6002 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:11:09.643229 6002 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:11:09.643380 6002 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:11:09.651614 6002 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:11:09.651794 6002 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:11:09.651929 6002 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 19:11:09.652055 6002 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.131
I20260811 19:11:09.662869 6002 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:38581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab
--principal=kudu/127.3.61.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.61.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.61.190:42953
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.131
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:11:09.664326 6002 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:11:09.665958 6002 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:11:09.680985 6009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:09.681054 6010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:09.681031 6012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:11:09.682346 6002 server_base.cc:1034] running on GCE node
I20260811 19:11:09.683066 6002 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:11:09.684084 6002 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:11:09.685354 6002 hybrid_clock.cc:648] HybridClock initialized: now 1786475469685202 us; error 105 us; skew 500 ppm
I20260811 19:11:09.685719 6002 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:11:09 dist-test-slave-t7g5 krb5kdc[5220](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475469, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:11:09.689355 6002 init.cc:377] Logged in from keytab as kudu/127.3.61.131@KRBTEST.COM (short username kudu)
I20260811 19:11:09.691205 6002 webserver.cc:460] Webserver started at http://127.3.61.131:41873/ using document root <none> and password file <none>
I20260811 19:11:09.691991 6002 fs_manager.cc:359] Metadata directory not provided
I20260811 19:11:09.692107 6002 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:11:09.692435 6002 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:11:09.696043 6002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/ts-2/data/instance:
uuid: "2e855b00e2984a8f986aaf94518ab798"
format_stamp: "Formatted at 2026-08-11 19:11:09 on dist-test-slave-t7g5"
server_key: "61546258b96c81a6a500d4846ab4c0b4"
server_key_iv: "311e938ae9004c9bd95659949964eccf"
server_key_version: "encryptionkey@0"
I20260811 19:11:09.696909 6002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/ts-2/wal/instance:
uuid: "2e855b00e2984a8f986aaf94518ab798"
format_stamp: "Formatted at 2026-08-11 19:11:09 on dist-test-slave-t7g5"
server_key: "61546258b96c81a6a500d4846ab4c0b4"
server_key_iv: "311e938ae9004c9bd95659949964eccf"
server_key_version: "encryptionkey@0"
I20260811 19:11:09.703058 6002 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260811 19:11:09.707683 6022 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:09.709506 6002 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260811 19:11:09.709760 6002 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/ts-2/wal
uuid: "2e855b00e2984a8f986aaf94518ab798"
format_stamp: "Formatted at 2026-08-11 19:11:09 on dist-test-slave-t7g5"
server_key: "61546258b96c81a6a500d4846ab4c0b4"
server_key_iv: "311e938ae9004c9bd95659949964eccf"
server_key_version: "encryptionkey@0"
I20260811 19:11:09.710028 6002 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:11:09.734825 6002 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:11:09.739642 6002 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:11:09.739933 6002 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:11:09.743705 6002 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:11:09.743832 6002 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:09.743984 6002 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:11:09.744065 6002 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 19:11:09 dist-test-slave-t7g5 krb5kdc[5220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475469, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.131@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:11:09.772049 6042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:09.745053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:44301 (local address 127.3.61.190:42953)
0811 19:11:09.745529 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:09.745534 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:09.747865 (+ 2331us) server_negotiation.cc:407] Connection header received
0811 19:11:09.750943 (+ 3078us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:09.750951 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:09.751062 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:09.751207 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:09.753805 (+ 2598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:09.755126 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:09.756933 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:09.757628 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:09.757796 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:09.764677 (+ 6881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:09.764684 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:09.764687 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:09.764727 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:09.767822 (+ 3095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:09.768920 (+ 1098us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:09.768925 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:09.768927 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:09.769076 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:09.769547 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:09.769551 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:09.769553 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:09.769908 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:09.770058 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:09.771226 (+ 1168us) server_negotiation.cc:299] Negotiation successful
0811 19:11:09.771561 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":247,"threads_started":1}
I20260811 19:11:09.773111 6044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:09.745928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:42953 (local address 127.3.61.131:44301)
0811 19:11:09.747519 (+ 1591us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:09.747637 (+ 118us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:09.750591 (+ 2954us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:09.751595 (+ 1004us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:09.751604 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:09.752504 (+ 900us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:11:09.753468 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:09.753501 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:09.755348 (+ 1847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:09.755353 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:09.756777 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:09.756794 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:09.758329 (+ 1535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:09.758350 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:09.758671 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:09.759780 (+ 1109us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:11:09.759823 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:11:09.764353 (+ 4530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:11:09.768022 (+ 3669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:09.768034 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:09.768059 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:09.768756 (+ 697us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:09.769193 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:09.769198 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:09.769201 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:09.769424 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:09.770049 (+ 625us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:11:09.770059 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:11:09.770640 (+ 581us) client_negotiation.cc:769] Sending connection context
0811 19:11:09.771057 (+ 417us) client_negotiation.cc:240] Negotiation successful
0811 19:11:09.771704 (+ 647us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1021,"thread_start_us":576,"threads_started":1}
I20260811 19:11:09.801467 6182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:09.784517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:42953 (local address 127.3.61.131:57623)
0811 19:11:09.785229 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:09.785250 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:09.785386 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:09.785871 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:09.785878 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:09.786892 (+ 1014us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:11:09.787868 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:09.787891 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:09.789238 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:09.789243 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:09.790925 (+ 1682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:09.790944 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:09.791741 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:09.791754 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:09.791895 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:09.792848 (+ 953us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:11:09.792870 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:11:09.793796 (+ 926us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:11:09.798002 (+ 4206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:09.798008 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:09.798013 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:09.798629 (+ 616us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:09.799155 (+ 526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:09.799160 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:09.799164 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:09.799359 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:09.800151 (+ 792us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:11:09.800155 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:11:09.800396 (+ 241us) client_negotiation.cc:769] Sending connection context
0811 19:11:09.800609 (+ 213us) client_negotiation.cc:240] Negotiation successful
0811 19:11:09.801042 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":410,"threads_started":1}
I20260811 19:11:09.801752 6042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:09.784609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:57623 (local address 127.3.61.190:42953)
0811 19:11:09.784904 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:09.784911 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:09.785332 (+ 421us) server_negotiation.cc:407] Connection header received
0811 19:11:09.785534 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:09.785540 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:09.785638 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:09.785782 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:09.788059 (+ 2277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:09.789067 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:09.791083 (+ 2016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:09.791531 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:09.791647 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:09.794016 (+ 2369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:09.794022 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:09.794025 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:09.794058 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:09.797781 (+ 3723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:09.798833 (+ 1052us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:09.798839 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:09.798842 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:09.798997 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:09.799548 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:09.799554 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:09.799557 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:09.799929 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:09.800093 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:09.801055 (+ 962us) server_negotiation.cc:299] Negotiation successful
0811 19:11:09.801342 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 19:11:09.805488 6002 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.131:37461
I20260811 19:11:09.805995 6270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.131:37461 every 8 connection(s)
I20260811 19:11:09.807364 6002 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
I20260811 19:11:09.809535 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 6002
I20260811 19:11:09.809772 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/ts-2/wal/instance
I20260811 19:11:09.810307 3318 external_mini_cluster.cc:1428] Setting key 4b7e48729346ab8c8f2afeae409eea9e
I20260811 19:11:09.810494 6271 heartbeater.cc:347] Connected to a master server at 127.3.61.190:42953
I20260811 19:11:09.810750 6271 heartbeater.cc:464] Registering TS with master...
I20260811 19:11:09.811632 6271 heartbeater.cc:511] Master 127.3.61.190:42953 requested a full tablet report, sending...
I20260811 19:11:09.813416 5298 ts_manager.cc:194] Registered new tserver with Master: 2e855b00e2984a8f986aaf94518ab798 (127.3.61.131:37461)
I20260811 19:11:09.814435 5298 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.131@KRBTEST.COM'} at 127.3.61.131:44301
I20260811 19:11:09.827885 3318 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 19:11:09 dist-test-slave-t7g5 krb5kdc[5220](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475469, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 19:11:09 dist-test-slave-t7g5 krb5kdc[5220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475469, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:11:09.895900 6042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:09.878034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53498 (local address 127.3.61.190:42953)
0811 19:11:09.878319 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:09.878327 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:09.878650 (+ 323us) server_negotiation.cc:407] Connection header received
0811 19:11:09.878885 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:09.878890 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:09.878959 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:09.879051 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:09.880945 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:09.881850 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:09.883607 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:09.884071 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:09.884205 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:09.888601 (+ 4396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:09.888607 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:09.888611 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:09.888652 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:09.892371 (+ 3719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:09.893312 (+ 941us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:09.893319 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:09.893321 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:09.893496 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:09.894049 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:09.894055 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:09.894057 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:09.894488 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:09.894906 (+ 418us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:09.895226 (+ 320us) server_negotiation.cc:299] Negotiation successful
0811 19:11:09.895503 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260811 19:11:09.901623 5298 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53498:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 19:11:09.909718 5298 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:11:09.936058 6292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:09.924156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:42915 (local address 127.0.0.1:56752)
0811 19:11:09.924947 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:09.925001 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:09.925215 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:09.926540 (+ 1325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:09.926550 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:09.926586 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:11:09.927077 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:09.927092 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:09.929570 (+ 2478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:09.929577 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:09.932314 (+ 2737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:09.932336 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:09.934865 (+ 2529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:09.934872 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:09.935140 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:09.935209 (+ 69us) client_negotiation.cc:769] Sending connection context
0811 19:11:09.935353 (+ 144us) client_negotiation.cc:240] Negotiation successful
0811 19:11:09.935615 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":356,"threads_started":1}
I20260811 19:11:09.938835 6293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:09.924476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56752 (local address 127.3.61.129:42915)
0811 19:11:09.925823 (+ 1347us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:09.925843 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:09.925874 (+ 31us) server_negotiation.cc:407] Connection header received
0811 19:11:09.925978 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:09.925986 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:09.926274 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:09.927305 (+ 1031us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:11:09.927516 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:09.929379 (+ 1863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:09.932795 (+ 3416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:09.934558 (+ 1763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:09.937179 (+ 2621us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:09.938191 (+ 1012us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:09.938294 (+ 103us) server_negotiation.cc:299] Negotiation successful
0811 19:11:09.938513 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1149,"thread_start_us":1017,"threads_started":1}
I20260811 19:11:09.943382 5546 tablet_service.cc:1467] Processing CreateTablet for tablet 17f7f0ef70d24b0a93e5e435b902611b (DEFAULT_TABLE table=test-table [id=8312af0c84974efc8fc5210736655fa7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:11:09.945256 5546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17f7f0ef70d24b0a93e5e435b902611b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:11:09.945852 6292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:09.925985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.130:41691 (local address 127.0.0.1:39212)
0811 19:11:09.936420 (+ 10435us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:09.936443 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:09.936573 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:09.938438 (+ 1865us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:09.938449 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:09.938483 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:11:09.938691 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:09.938701 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:09.940822 (+ 2121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:09.940828 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:09.942747 (+ 1919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:09.942764 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:09.944681 (+ 1917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:09.944688 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:09.945099 (+ 411us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:09.945138 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 19:11:09.945233 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 19:11:09.945491 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10298,"thread_start_us":11833,"threads_started":1}
I20260811 19:11:09.946583 6298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:09.926650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39212 (local address 127.3.61.130:41691)
0811 19:11:09.937680 (+ 11030us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:09.937691 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:09.937727 (+ 36us) server_negotiation.cc:407] Connection header received
0811 19:11:09.937855 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:09.937864 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:09.938172 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:09.938899 (+ 727us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:11:09.939262 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:09.940593 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:09.942958 (+ 2365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:09.944473 (+ 1515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:09.944633 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:09.944973 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:09.946107 (+ 1134us) server_negotiation.cc:299] Negotiation successful
0811 19:11:09.946301 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10845,"thread_start_us":10710,"threads_started":1}
I20260811 19:11:09.948011 6294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:09.925545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.131:37461 (local address 127.0.0.1:32872)
0811 19:11:09.926187 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:09.926206 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:09.926325 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:09.940498 (+ 14173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:09.940504 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:09.940531 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:11:09.940961 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:09.940971 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:09.943277 (+ 2306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:09.943283 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:09.944742 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:09.944760 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:09.947240 (+ 2480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:09.947247 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:09.947420 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:09.947456 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 19:11:09.947527 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 19:11:09.947677 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":1275,"threads_started":1}
I20260811 19:11:09.948756 6297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:09.926312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32872 (local address 127.3.61.131:37461)
0811 19:11:09.939928 (+ 13616us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:09.939937 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:09.939973 (+ 36us) server_negotiation.cc:407] Connection header received
0811 19:11:09.940097 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:09.940106 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:09.940294 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:09.940498 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:11:09.941150 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:09.943166 (+ 2016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:09.944965 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:09.946977 (+ 2012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:09.947944 (+ 967us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:09.948179 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:09.948262 (+ 83us) server_negotiation.cc:299] Negotiation successful
0811 19:11:09.948428 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13438,"thread_start_us":13263,"threads_started":1}
I20260811 19:11:09.950481 5841 tablet_service.cc:1467] Processing CreateTablet for tablet 17f7f0ef70d24b0a93e5e435b902611b (DEFAULT_TABLE table=test-table [id=8312af0c84974efc8fc5210736655fa7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:11:09.952790 5841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17f7f0ef70d24b0a93e5e435b902611b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:11:09.952431 6124 tablet_service.cc:1467] Processing CreateTablet for tablet 17f7f0ef70d24b0a93e5e435b902611b (DEFAULT_TABLE table=test-table [id=8312af0c84974efc8fc5210736655fa7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:11:09.954037 6124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17f7f0ef70d24b0a93e5e435b902611b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:11:09.959220 6303 tablet_bootstrap.cc:492] T 17f7f0ef70d24b0a93e5e435b902611b P 02c4a487189f44b69b7f49f33a486807: Bootstrap starting.
I20260811 19:11:09.962625 6303 tablet_bootstrap.cc:654] T 17f7f0ef70d24b0a93e5e435b902611b P 02c4a487189f44b69b7f49f33a486807: Neither blocks nor log segments found. Creating new log.
I20260811 19:11:09.963725 6303 log.cc:824] T 17f7f0ef70d24b0a93e5e435b902611b P 02c4a487189f44b69b7f49f33a486807: Log is configured to *not* fsync() on all Append() calls
I20260811 19:11:09.966730 6303 tablet_bootstrap.cc:492] T 17f7f0ef70d24b0a93e5e435b902611b P 02c4a487189f44b69b7f49f33a486807: No bootstrap required, opened a new log
I20260811 19:11:09.967145 6303 ts_tablet_manager.cc:1397] T 17f7f0ef70d24b0a93e5e435b902611b P 02c4a487189f44b69b7f49f33a486807: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.005s
I20260811 19:11:09.969278 6309 tablet_bootstrap.cc:492] T 17f7f0ef70d24b0a93e5e435b902611b P 2e855b00e2984a8f986aaf94518ab798: Bootstrap starting.
I20260811 19:11:09.970321 6310 tablet_bootstrap.cc:492] T 17f7f0ef70d24b0a93e5e435b902611b P df6626ebdf524b78a3551525563ac21d: Bootstrap starting.
I20260811 19:11:09.973618 6309 tablet_bootstrap.cc:654] T 17f7f0ef70d24b0a93e5e435b902611b P 2e855b00e2984a8f986aaf94518ab798: Neither blocks nor log segments found. Creating new log.
I20260811 19:11:09.973919 6310 tablet_bootstrap.cc:654] T 17f7f0ef70d24b0a93e5e435b902611b P df6626ebdf524b78a3551525563ac21d: Neither blocks nor log segments found. Creating new log.
I20260811 19:11:09.975342 6309 log.cc:824] T 17f7f0ef70d24b0a93e5e435b902611b P 2e855b00e2984a8f986aaf94518ab798: Log is configured to *not* fsync() on all Append() calls
I20260811 19:11:09.975339 6310 log.cc:824] T 17f7f0ef70d24b0a93e5e435b902611b P df6626ebdf524b78a3551525563ac21d: Log is configured to *not* fsync() on all Append() calls
I20260811 19:11:09.978049 6303 raft_consensus.cc:348] T 17f7f0ef70d24b0a93e5e435b902611b P 02c4a487189f44b69b7f49f33a486807 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c4a487189f44b69b7f49f33a486807" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 42915 } } peers { permanent_uuid: "2e855b00e2984a8f986aaf94518ab798" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37461 } } peers { permanent_uuid: "df6626ebdf524b78a3551525563ac21d" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41691 } }
I20260811 19:11:09.978439 6303 raft_consensus.cc:374] T 17f7f0ef70d24b0a93e5e435b902611b P 02c4a487189f44b69b7f49f33a486807 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:11:09.978555 6303 raft_consensus.cc:729] T 17f7f0ef70d24b0a93e5e435b902611b P 02c4a487189f44b69b7f49f33a486807 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02c4a487189f44b69b7f49f33a486807, State: Initialized, Role: FOLLOWER
I20260811 19:11:09.979192 6303 consensus_queue.cc:260] T 17f7f0ef70d24b0a93e5e435b902611b P 02c4a487189f44b69b7f49f33a486807 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c4a487189f44b69b7f49f33a486807" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 42915 } } peers { permanent_uuid: "2e855b00e2984a8f986aaf94518ab798" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37461 } } peers { permanent_uuid: "df6626ebdf524b78a3551525563ac21d" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41691 } }
I20260811 19:11:09.980589 6310 tablet_bootstrap.cc:492] T 17f7f0ef70d24b0a93e5e435b902611b P df6626ebdf524b78a3551525563ac21d: No bootstrap required, opened a new log
I20260811 19:11:09.981172 6303 ts_tablet_manager.cc:1428] T 17f7f0ef70d24b0a93e5e435b902611b P 02c4a487189f44b69b7f49f33a486807: Time spent starting tablet: real 0.014s user 0.007s sys 0.005s
I20260811 19:11:09.981202 6310 ts_tablet_manager.cc:1397] T 17f7f0ef70d24b0a93e5e435b902611b P df6626ebdf524b78a3551525563ac21d: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.007s
I20260811 19:11:09.984077 6309 tablet_bootstrap.cc:492] T 17f7f0ef70d24b0a93e5e435b902611b P 2e855b00e2984a8f986aaf94518ab798: No bootstrap required, opened a new log
I20260811 19:11:09.984352 6309 ts_tablet_manager.cc:1397] T 17f7f0ef70d24b0a93e5e435b902611b P 2e855b00e2984a8f986aaf94518ab798: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.012s
I20260811 19:11:09.992363 6310 raft_consensus.cc:348] T 17f7f0ef70d24b0a93e5e435b902611b P df6626ebdf524b78a3551525563ac21d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c4a487189f44b69b7f49f33a486807" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 42915 } } peers { permanent_uuid: "2e855b00e2984a8f986aaf94518ab798" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37461 } } peers { permanent_uuid: "df6626ebdf524b78a3551525563ac21d" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41691 } }
I20260811 19:11:09.992911 6310 raft_consensus.cc:374] T 17f7f0ef70d24b0a93e5e435b902611b P df6626ebdf524b78a3551525563ac21d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:11:09.993036 6310 raft_consensus.cc:729] T 17f7f0ef70d24b0a93e5e435b902611b P df6626ebdf524b78a3551525563ac21d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df6626ebdf524b78a3551525563ac21d, State: Initialized, Role: FOLLOWER
I20260811 19:11:09.993778 6310 consensus_queue.cc:260] T 17f7f0ef70d24b0a93e5e435b902611b P df6626ebdf524b78a3551525563ac21d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c4a487189f44b69b7f49f33a486807" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 42915 } } peers { permanent_uuid: "2e855b00e2984a8f986aaf94518ab798" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37461 } } peers { permanent_uuid: "df6626ebdf524b78a3551525563ac21d" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41691 } }
I20260811 19:11:09.994517 6309 raft_consensus.cc:348] T 17f7f0ef70d24b0a93e5e435b902611b P 2e855b00e2984a8f986aaf94518ab798 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c4a487189f44b69b7f49f33a486807" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 42915 } } peers { permanent_uuid: "2e855b00e2984a8f986aaf94518ab798" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37461 } } peers { permanent_uuid: "df6626ebdf524b78a3551525563ac21d" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41691 } }
I20260811 19:11:09.994958 6309 raft_consensus.cc:374] T 17f7f0ef70d24b0a93e5e435b902611b P 2e855b00e2984a8f986aaf94518ab798 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:11:09.994963 5994 heartbeater.cc:503] Master 127.3.61.190:42953 was elected leader, sending a full tablet report...
I20260811 19:11:09.995072 6309 raft_consensus.cc:729] T 17f7f0ef70d24b0a93e5e435b902611b P 2e855b00e2984a8f986aaf94518ab798 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e855b00e2984a8f986aaf94518ab798, State: Initialized, Role: FOLLOWER
I20260811 19:11:09.995807 6310 ts_tablet_manager.cc:1428] T 17f7f0ef70d24b0a93e5e435b902611b P df6626ebdf524b78a3551525563ac21d: Time spent starting tablet: real 0.014s user 0.010s sys 0.003s
I20260811 19:11:09.995718 6309 consensus_queue.cc:260] T 17f7f0ef70d24b0a93e5e435b902611b P 2e855b00e2984a8f986aaf94518ab798 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c4a487189f44b69b7f49f33a486807" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 42915 } } peers { permanent_uuid: "2e855b00e2984a8f986aaf94518ab798" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37461 } } peers { permanent_uuid: "df6626ebdf524b78a3551525563ac21d" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41691 } }
I20260811 19:11:09.997231 6271 heartbeater.cc:503] Master 127.3.61.190:42953 was elected leader, sending a full tablet report...
I20260811 19:11:09.997519 6309 ts_tablet_manager.cc:1428] T 17f7f0ef70d24b0a93e5e435b902611b P 2e855b00e2984a8f986aaf94518ab798: Time spent starting tablet: real 0.013s user 0.011s sys 0.003s
W20260811 19:11:10.059514 6272 tablet.cc:2369] T 17f7f0ef70d24b0a93e5e435b902611b P 2e855b00e2984a8f986aaf94518ab798: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:11:10.084307 6314 raft_consensus.cc:482] T 17f7f0ef70d24b0a93e5e435b902611b P 02c4a487189f44b69b7f49f33a486807 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:11:10.084722 6314 raft_consensus.cc:504] T 17f7f0ef70d24b0a93e5e435b902611b P 02c4a487189f44b69b7f49f33a486807 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c4a487189f44b69b7f49f33a486807" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 42915 } } peers { permanent_uuid: "2e855b00e2984a8f986aaf94518ab798" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37461 } } peers { permanent_uuid: "df6626ebdf524b78a3551525563ac21d" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41691 } }
I20260811 19:11:10.087275 6314 leader_election.cc:288] T 17f7f0ef70d24b0a93e5e435b902611b P 02c4a487189f44b69b7f49f33a486807 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2e855b00e2984a8f986aaf94518ab798 (127.3.61.131:37461), df6626ebdf524b78a3551525563ac21d (127.3.61.130:41691)
I20260811 19:11:10.096268 6297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:10.087842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:42729 (local address 127.3.61.131:37461)
0811 19:11:10.088093 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:10.088099 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:10.088919 (+ 820us) server_negotiation.cc:407] Connection header received
0811 19:11:10.089142 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:10.089146 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:10.089212 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:10.089316 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:11:10.089976 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:10.091123 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:10.093978 (+ 2855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:10.095219 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:10.095385 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:10.095466 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:10.095941 (+ 475us) server_negotiation.cc:299] Negotiation successful
0811 19:11:10.096054 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260811 19:11:10.096546 6318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:10.087853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.131:37461 (local address 127.3.61.129:42729)
0811 19:11:10.088721 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:10.088750 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:10.088978 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:10.089365 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:10.089369 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:10.089402 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:11:10.089802 (+ 400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:10.089815 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:10.091293 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:10.091298 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:10.093782 (+ 2484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:10.093796 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:10.095521 (+ 1725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:10.095529 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:10.095712 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:10.095752 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 19:11:10.095874 (+ 122us) client_negotiation.cc:240] Negotiation successful
0811 19:11:10.096032 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":749,"mutex_wait_us":70,"thread_start_us":949,"threads_started":1}
I20260811 19:11:10.096769 6298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:10.087876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:37637 (local address 127.3.61.130:41691)
0811 19:11:10.088093 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:10.088099 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:10.089253 (+ 1154us) server_negotiation.cc:407] Connection header received
0811 19:11:10.089411 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:10.089417 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:10.089493 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:10.089561 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:11:10.090215 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:10.091317 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:10.093967 (+ 2650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:10.095192 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:10.095385 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:10.095488 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:10.096363 (+ 875us) server_negotiation.cc:299] Negotiation successful
0811 19:11:10.096515 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 19:11:10.096912 6317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:10.087853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.130:41691 (local address 127.3.61.129:37637)
0811 19:11:10.089112 (+ 1259us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:10.089127 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:10.089276 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:10.089601 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:10.089606 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:10.089633 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:11:10.090035 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:10.090044 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:10.091465 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:10.091470 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:10.093772 (+ 2302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:10.093796 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:10.095938 (+ 2142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:10.095947 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:10.096135 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:10.096173 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 19:11:10.096287 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 19:11:10.096425 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1126,"mutex_wait_us":80,"thread_start_us":522,"threads_started":1}
I20260811 19:11:10.097904 6175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17f7f0ef70d24b0a93e5e435b902611b" candidate_uuid: "02c4a487189f44b69b7f49f33a486807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e855b00e2984a8f986aaf94518ab798" is_pre_election: true
I20260811 19:11:10.098101 5891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17f7f0ef70d24b0a93e5e435b902611b" candidate_uuid: "02c4a487189f44b69b7f49f33a486807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df6626ebdf524b78a3551525563ac21d" is_pre_election: true
I20260811 19:11:10.098675 6175 raft_consensus.cc:2454] T 17f7f0ef70d24b0a93e5e435b902611b P 2e855b00e2984a8f986aaf94518ab798 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02c4a487189f44b69b7f49f33a486807 in term 0.
I20260811 19:11:10.098675 5891 raft_consensus.cc:2454] T 17f7f0ef70d24b0a93e5e435b902611b P df6626ebdf524b78a3551525563ac21d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02c4a487189f44b69b7f49f33a486807 in term 0.
I20260811 19:11:10.099593 5461 leader_election.cc:302] T 17f7f0ef70d24b0a93e5e435b902611b P 02c4a487189f44b69b7f49f33a486807 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c4a487189f44b69b7f49f33a486807, df6626ebdf524b78a3551525563ac21d; no voters:
I20260811 19:11:10.100122 6314 raft_consensus.cc:2781] T 17f7f0ef70d24b0a93e5e435b902611b P 02c4a487189f44b69b7f49f33a486807 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:11:10.100311 6314 raft_consensus.cc:482] T 17f7f0ef70d24b0a93e5e435b902611b P 02c4a487189f44b69b7f49f33a486807 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:11:10.100419 6314 raft_consensus.cc:3037] T 17f7f0ef70d24b0a93e5e435b902611b P 02c4a487189f44b69b7f49f33a486807 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:11:10.102726 6314 raft_consensus.cc:504] T 17f7f0ef70d24b0a93e5e435b902611b P 02c4a487189f44b69b7f49f33a486807 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c4a487189f44b69b7f49f33a486807" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 42915 } } peers { permanent_uuid: "2e855b00e2984a8f986aaf94518ab798" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37461 } } peers { permanent_uuid: "df6626ebdf524b78a3551525563ac21d" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41691 } }
I20260811 19:11:10.103382 6314 leader_election.cc:288] T 17f7f0ef70d24b0a93e5e435b902611b P 02c4a487189f44b69b7f49f33a486807 [CANDIDATE]: Term 1 election: Requested vote from peers 2e855b00e2984a8f986aaf94518ab798 (127.3.61.131:37461), df6626ebdf524b78a3551525563ac21d (127.3.61.130:41691)
I20260811 19:11:10.103958 5891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17f7f0ef70d24b0a93e5e435b902611b" candidate_uuid: "02c4a487189f44b69b7f49f33a486807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df6626ebdf524b78a3551525563ac21d"
I20260811 19:11:10.104022 6175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17f7f0ef70d24b0a93e5e435b902611b" candidate_uuid: "02c4a487189f44b69b7f49f33a486807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e855b00e2984a8f986aaf94518ab798"
I20260811 19:11:10.104179 5891 raft_consensus.cc:3037] T 17f7f0ef70d24b0a93e5e435b902611b P df6626ebdf524b78a3551525563ac21d [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:11:10.104216 6175 raft_consensus.cc:3037] T 17f7f0ef70d24b0a93e5e435b902611b P 2e855b00e2984a8f986aaf94518ab798 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:11:10.106678 6175 raft_consensus.cc:2454] T 17f7f0ef70d24b0a93e5e435b902611b P 2e855b00e2984a8f986aaf94518ab798 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02c4a487189f44b69b7f49f33a486807 in term 1.
I20260811 19:11:10.106701 5891 raft_consensus.cc:2454] T 17f7f0ef70d24b0a93e5e435b902611b P df6626ebdf524b78a3551525563ac21d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02c4a487189f44b69b7f49f33a486807 in term 1.
I20260811 19:11:10.107168 5460 leader_election.cc:302] T 17f7f0ef70d24b0a93e5e435b902611b P 02c4a487189f44b69b7f49f33a486807 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c4a487189f44b69b7f49f33a486807, 2e855b00e2984a8f986aaf94518ab798; no voters:
I20260811 19:11:10.107512 6314 raft_consensus.cc:2781] T 17f7f0ef70d24b0a93e5e435b902611b P 02c4a487189f44b69b7f49f33a486807 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:11:10.108107 6314 raft_consensus.cc:686] T 17f7f0ef70d24b0a93e5e435b902611b P 02c4a487189f44b69b7f49f33a486807 [term 1 LEADER]: Becoming Leader. State: Replica: 02c4a487189f44b69b7f49f33a486807, State: Running, Role: LEADER
I20260811 19:11:10.108722 6314 consensus_queue.cc:237] T 17f7f0ef70d24b0a93e5e435b902611b P 02c4a487189f44b69b7f49f33a486807 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c4a487189f44b69b7f49f33a486807" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 42915 } } peers { permanent_uuid: "2e855b00e2984a8f986aaf94518ab798" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37461 } } peers { permanent_uuid: "df6626ebdf524b78a3551525563ac21d" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41691 } }
I20260811 19:11:10.114334 5300 catalog_manager.cc:5495] T 17f7f0ef70d24b0a93e5e435b902611b P 02c4a487189f44b69b7f49f33a486807 reported cstate change: term changed from 0 to 1, leader changed from <none> to 02c4a487189f44b69b7f49f33a486807 (127.3.61.129). New cstate: current_term: 1 leader_uuid: "02c4a487189f44b69b7f49f33a486807" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c4a487189f44b69b7f49f33a486807" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 42915 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2e855b00e2984a8f986aaf94518ab798" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df6626ebdf524b78a3551525563ac21d" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41691 } health_report { overall_health: UNKNOWN } } }
W20260811 19:11:10.134030 5705 tablet.cc:2369] T 17f7f0ef70d24b0a93e5e435b902611b P 02c4a487189f44b69b7f49f33a486807: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:11:10.156201 6293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:10.148582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56754 (local address 127.3.61.129:42915)
0811 19:11:10.148829 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:10.148836 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:10.148914 (+ 78us) server_negotiation.cc:407] Connection header received
0811 19:11:10.149356 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:10.149361 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:10.149474 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:10.149615 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:11:10.150120 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:10.150868 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:10.152619 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:10.153083 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:10.153230 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:10.154399 (+ 1169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:11:10.155291 (+ 892us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:11:10.155445 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:10.155714 (+ 269us) server_negotiation.cc:299] Negotiation successful
0811 19:11:10.155826 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 19:11:10.167294 5891 raft_consensus.cc:1261] T 17f7f0ef70d24b0a93e5e435b902611b P df6626ebdf524b78a3551525563ac21d [term 1 FOLLOWER]: Refusing update from remote peer 02c4a487189f44b69b7f49f33a486807: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:11:10.167286 6175 raft_consensus.cc:1261] T 17f7f0ef70d24b0a93e5e435b902611b P 2e855b00e2984a8f986aaf94518ab798 [term 1 FOLLOWER]: Refusing update from remote peer 02c4a487189f44b69b7f49f33a486807: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:11:10.168911 6319 consensus_queue.cc:1035] T 17f7f0ef70d24b0a93e5e435b902611b P 02c4a487189f44b69b7f49f33a486807 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e855b00e2984a8f986aaf94518ab798" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:11:10.169469 6314 consensus_queue.cc:1035] T 17f7f0ef70d24b0a93e5e435b902611b P 02c4a487189f44b69b7f49f33a486807 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df6626ebdf524b78a3551525563ac21d" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:11:10.191496 6322 mvcc.cc:205] Tried to move back new op lower bound from 7317403525792223232 to 7317403525570027520. Current Snapshot: MvccSnapshot[applied={T|T < 7317403525792223232}]
I20260811 19:11:10.193187 6326 mvcc.cc:205] Tried to move back new op lower bound from 7317403525792223232 to 7317403525570027520. Current Snapshot: MvccSnapshot[applied={T|T < 7317403525792223232}]
I20260811 19:11:10.217545 5300 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53498:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:11:10.218006 5300 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53498:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:11:10.222114 5300 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6d3b329daf2c495a81783be32467ea40: Sending DeleteTablet for 3 replicas of tablet 17f7f0ef70d24b0a93e5e435b902611b
I20260811 19:11:10.223845 6124 tablet_service.cc:1514] Processing DeleteTablet for tablet 17f7f0ef70d24b0a93e5e435b902611b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:11:10 UTC) from {username='kudu', principal='kudu/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:32872
I20260811 19:11:10.224252 5841 tablet_service.cc:1514] Processing DeleteTablet for tablet 17f7f0ef70d24b0a93e5e435b902611b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:11:10 UTC) from {username='kudu', principal='kudu/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:39212
I20260811 19:11:10.225517 6343 tablet_replica.cc:330] T 17f7f0ef70d24b0a93e5e435b902611b P df6626ebdf524b78a3551525563ac21d: stopping tablet replica
I20260811 19:11:10.226048 5546 tablet_service.cc:1514] Processing DeleteTablet for tablet 17f7f0ef70d24b0a93e5e435b902611b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:11:10 UTC) from {username='kudu', principal='kudu/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:56752
I20260811 19:11:10.226343 6343 raft_consensus.cc:2229] T 17f7f0ef70d24b0a93e5e435b902611b P df6626ebdf524b78a3551525563ac21d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:11:10.226954 6343 raft_consensus.cc:2258] T 17f7f0ef70d24b0a93e5e435b902611b P df6626ebdf524b78a3551525563ac21d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:11:10.229439 6343 ts_tablet_manager.cc:1905] T 17f7f0ef70d24b0a93e5e435b902611b P df6626ebdf524b78a3551525563ac21d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:11:10.228976 5300 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53498:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 19:11:10.230899 5300 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:11:10.237478 6343 ts_tablet_manager.cc:1918] T 17f7f0ef70d24b0a93e5e435b902611b P df6626ebdf524b78a3551525563ac21d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:11:10.237918 6343 log.cc:1196] T 17f7f0ef70d24b0a93e5e435b902611b P df6626ebdf524b78a3551525563ac21d: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/ts-1/wal/wals/17f7f0ef70d24b0a93e5e435b902611b
I20260811 19:11:10.238656 6343 ts_tablet_manager.cc:1939] T 17f7f0ef70d24b0a93e5e435b902611b P df6626ebdf524b78a3551525563ac21d: Deleting consensus metadata
I20260811 19:11:10.241504 5274 catalog_manager.cc:4841] TS df6626ebdf524b78a3551525563ac21d (127.3.61.130:41691): tablet 17f7f0ef70d24b0a93e5e435b902611b (table test-table [id=8312af0c84974efc8fc5210736655fa7]) successfully deleted
I20260811 19:11:10.242452 5544 tablet_service.cc:1467] Processing CreateTablet for tablet b2ae87b5f3d249f48c61b49b5f5fbc77 (DEFAULT_TABLE table=test-table [id=12fa231950d944fd8fcbdc4852e747a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:11:10.242471 6123 tablet_service.cc:1467] Processing CreateTablet for tablet b2ae87b5f3d249f48c61b49b5f5fbc77 (DEFAULT_TABLE table=test-table [id=12fa231950d944fd8fcbdc4852e747a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:11:10.243089 6123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2ae87b5f3d249f48c61b49b5f5fbc77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:11:10.243099 5544 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2ae87b5f3d249f48c61b49b5f5fbc77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:11:10.246634 5841 tablet_service.cc:1467] Processing CreateTablet for tablet b2ae87b5f3d249f48c61b49b5f5fbc77 (DEFAULT_TABLE table=test-table [id=12fa231950d944fd8fcbdc4852e747a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:11:10.247315 5841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2ae87b5f3d249f48c61b49b5f5fbc77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:11:10.251461 6303 tablet_bootstrap.cc:492] T b2ae87b5f3d249f48c61b49b5f5fbc77 P 02c4a487189f44b69b7f49f33a486807: Bootstrap starting.
I20260811 19:11:10.254478 6303 tablet_bootstrap.cc:654] T b2ae87b5f3d249f48c61b49b5f5fbc77 P 02c4a487189f44b69b7f49f33a486807: Neither blocks nor log segments found. Creating new log.
I20260811 19:11:10.261730 6309 tablet_bootstrap.cc:492] T b2ae87b5f3d249f48c61b49b5f5fbc77 P 2e855b00e2984a8f986aaf94518ab798: Bootstrap starting.
I20260811 19:11:10.263747 6309 tablet_bootstrap.cc:654] T b2ae87b5f3d249f48c61b49b5f5fbc77 P 2e855b00e2984a8f986aaf94518ab798: Neither blocks nor log segments found. Creating new log.
I20260811 19:11:10.266302 6310 tablet_bootstrap.cc:492] T b2ae87b5f3d249f48c61b49b5f5fbc77 P df6626ebdf524b78a3551525563ac21d: Bootstrap starting.
I20260811 19:11:10.268271 6310 tablet_bootstrap.cc:654] T b2ae87b5f3d249f48c61b49b5f5fbc77 P df6626ebdf524b78a3551525563ac21d: Neither blocks nor log segments found. Creating new log.
I20260811 19:11:10.270910 6344 tablet_replica.cc:330] T 17f7f0ef70d24b0a93e5e435b902611b P 02c4a487189f44b69b7f49f33a486807: stopping tablet replica
I20260811 19:11:10.271616 6344 raft_consensus.cc:2229] T 17f7f0ef70d24b0a93e5e435b902611b P 02c4a487189f44b69b7f49f33a486807 [term 1 LEADER]: Raft consensus shutting down.
I20260811 19:11:10.272513 6344 raft_consensus.cc:2258] T 17f7f0ef70d24b0a93e5e435b902611b P 02c4a487189f44b69b7f49f33a486807 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:11:10.273406 6342 tablet_replica.cc:330] T 17f7f0ef70d24b0a93e5e435b902611b P 2e855b00e2984a8f986aaf94518ab798: stopping tablet replica
I20260811 19:11:10.274271 6342 raft_consensus.cc:2229] T 17f7f0ef70d24b0a93e5e435b902611b P 2e855b00e2984a8f986aaf94518ab798 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:11:10.274865 6342 raft_consensus.cc:2258] T 17f7f0ef70d24b0a93e5e435b902611b P 2e855b00e2984a8f986aaf94518ab798 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:11:10.275442 6344 ts_tablet_manager.cc:1905] T 17f7f0ef70d24b0a93e5e435b902611b P 02c4a487189f44b69b7f49f33a486807: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:11:10.278352 6342 ts_tablet_manager.cc:1905] T 17f7f0ef70d24b0a93e5e435b902611b P 2e855b00e2984a8f986aaf94518ab798: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:11:10.284103 6344 ts_tablet_manager.cc:1918] T 17f7f0ef70d24b0a93e5e435b902611b P 02c4a487189f44b69b7f49f33a486807: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:11:10.284219 6310 tablet_bootstrap.cc:492] T b2ae87b5f3d249f48c61b49b5f5fbc77 P df6626ebdf524b78a3551525563ac21d: No bootstrap required, opened a new log
I20260811 19:11:10.284337 6344 log.cc:1196] T 17f7f0ef70d24b0a93e5e435b902611b P 02c4a487189f44b69b7f49f33a486807: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/ts-0/wal/wals/17f7f0ef70d24b0a93e5e435b902611b
I20260811 19:11:10.284394 6310 ts_tablet_manager.cc:1397] T b2ae87b5f3d249f48c61b49b5f5fbc77 P df6626ebdf524b78a3551525563ac21d: Time spent bootstrapping tablet: real 0.018s user 0.001s sys 0.008s
I20260811 19:11:10.284965 6344 ts_tablet_manager.cc:1939] T 17f7f0ef70d24b0a93e5e435b902611b P 02c4a487189f44b69b7f49f33a486807: Deleting consensus metadata
I20260811 19:11:10.285295 6310 raft_consensus.cc:348] T b2ae87b5f3d249f48c61b49b5f5fbc77 P df6626ebdf524b78a3551525563ac21d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c4a487189f44b69b7f49f33a486807" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 42915 } } peers { permanent_uuid: "df6626ebdf524b78a3551525563ac21d" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41691 } } peers { permanent_uuid: "2e855b00e2984a8f986aaf94518ab798" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37461 } }
I20260811 19:11:10.286648 6310 raft_consensus.cc:374] T b2ae87b5f3d249f48c61b49b5f5fbc77 P df6626ebdf524b78a3551525563ac21d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:11:10.286780 6310 raft_consensus.cc:729] T b2ae87b5f3d249f48c61b49b5f5fbc77 P df6626ebdf524b78a3551525563ac21d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df6626ebdf524b78a3551525563ac21d, State: Initialized, Role: FOLLOWER
I20260811 19:11:10.287137 5273 catalog_manager.cc:4841] TS 02c4a487189f44b69b7f49f33a486807 (127.3.61.129:42915): tablet 17f7f0ef70d24b0a93e5e435b902611b (table test-table [id=8312af0c84974efc8fc5210736655fa7]) successfully deleted
I20260811 19:11:10.287591 6310 consensus_queue.cc:260] T b2ae87b5f3d249f48c61b49b5f5fbc77 P df6626ebdf524b78a3551525563ac21d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c4a487189f44b69b7f49f33a486807" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 42915 } } peers { permanent_uuid: "df6626ebdf524b78a3551525563ac21d" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41691 } } peers { permanent_uuid: "2e855b00e2984a8f986aaf94518ab798" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37461 } }
I20260811 19:11:10.288421 6342 ts_tablet_manager.cc:1918] T 17f7f0ef70d24b0a93e5e435b902611b P 2e855b00e2984a8f986aaf94518ab798: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:11:10.288504 6310 ts_tablet_manager.cc:1428] T b2ae87b5f3d249f48c61b49b5f5fbc77 P df6626ebdf524b78a3551525563ac21d: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260811 19:11:10.288615 6342 log.cc:1196] T 17f7f0ef70d24b0a93e5e435b902611b P 2e855b00e2984a8f986aaf94518ab798: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/ts-2/wal/wals/17f7f0ef70d24b0a93e5e435b902611b
I20260811 19:11:10.289196 6342 ts_tablet_manager.cc:1939] T 17f7f0ef70d24b0a93e5e435b902611b P 2e855b00e2984a8f986aaf94518ab798: Deleting consensus metadata
I20260811 19:11:10.291326 6309 tablet_bootstrap.cc:492] T b2ae87b5f3d249f48c61b49b5f5fbc77 P 2e855b00e2984a8f986aaf94518ab798: No bootstrap required, opened a new log
I20260811 19:11:10.291461 6309 ts_tablet_manager.cc:1397] T b2ae87b5f3d249f48c61b49b5f5fbc77 P 2e855b00e2984a8f986aaf94518ab798: Time spent bootstrapping tablet: real 0.030s user 0.007s sys 0.002s
I20260811 19:11:10.291431 5273 catalog_manager.cc:4841] TS 2e855b00e2984a8f986aaf94518ab798 (127.3.61.131:37461): tablet 17f7f0ef70d24b0a93e5e435b902611b (table test-table [id=8312af0c84974efc8fc5210736655fa7]) successfully deleted
I20260811 19:11:10.292541 6309 raft_consensus.cc:348] T b2ae87b5f3d249f48c61b49b5f5fbc77 P 2e855b00e2984a8f986aaf94518ab798 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c4a487189f44b69b7f49f33a486807" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 42915 } } peers { permanent_uuid: "df6626ebdf524b78a3551525563ac21d" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41691 } } peers { permanent_uuid: "2e855b00e2984a8f986aaf94518ab798" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37461 } }
I20260811 19:11:10.292807 6309 raft_consensus.cc:374] T b2ae87b5f3d249f48c61b49b5f5fbc77 P 2e855b00e2984a8f986aaf94518ab798 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:11:10.292909 6309 raft_consensus.cc:729] T b2ae87b5f3d249f48c61b49b5f5fbc77 P 2e855b00e2984a8f986aaf94518ab798 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e855b00e2984a8f986aaf94518ab798, State: Initialized, Role: FOLLOWER
I20260811 19:11:10.293256 6309 consensus_queue.cc:260] T b2ae87b5f3d249f48c61b49b5f5fbc77 P 2e855b00e2984a8f986aaf94518ab798 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c4a487189f44b69b7f49f33a486807" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 42915 } } peers { permanent_uuid: "df6626ebdf524b78a3551525563ac21d" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41691 } } peers { permanent_uuid: "2e855b00e2984a8f986aaf94518ab798" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37461 } }
I20260811 19:11:10.294044 6309 ts_tablet_manager.cc:1428] T b2ae87b5f3d249f48c61b49b5f5fbc77 P 2e855b00e2984a8f986aaf94518ab798: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 19:11:10.294909 6303 tablet_bootstrap.cc:492] T b2ae87b5f3d249f48c61b49b5f5fbc77 P 02c4a487189f44b69b7f49f33a486807: No bootstrap required, opened a new log
I20260811 19:11:10.295112 6303 ts_tablet_manager.cc:1397] T b2ae87b5f3d249f48c61b49b5f5fbc77 P 02c4a487189f44b69b7f49f33a486807: Time spent bootstrapping tablet: real 0.044s user 0.008s sys 0.008s
I20260811 19:11:10.296105 6303 raft_consensus.cc:348] T b2ae87b5f3d249f48c61b49b5f5fbc77 P 02c4a487189f44b69b7f49f33a486807 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c4a487189f44b69b7f49f33a486807" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 42915 } } peers { permanent_uuid: "df6626ebdf524b78a3551525563ac21d" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41691 } } peers { permanent_uuid: "2e855b00e2984a8f986aaf94518ab798" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37461 } }
I20260811 19:11:10.296296 6303 raft_consensus.cc:374] T b2ae87b5f3d249f48c61b49b5f5fbc77 P 02c4a487189f44b69b7f49f33a486807 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:11:10.296373 6303 raft_consensus.cc:729] T b2ae87b5f3d249f48c61b49b5f5fbc77 P 02c4a487189f44b69b7f49f33a486807 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02c4a487189f44b69b7f49f33a486807, State: Initialized, Role: FOLLOWER
I20260811 19:11:10.296638 6303 consensus_queue.cc:260] T b2ae87b5f3d249f48c61b49b5f5fbc77 P 02c4a487189f44b69b7f49f33a486807 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c4a487189f44b69b7f49f33a486807" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 42915 } } peers { permanent_uuid: "df6626ebdf524b78a3551525563ac21d" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41691 } } peers { permanent_uuid: "2e855b00e2984a8f986aaf94518ab798" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37461 } }
I20260811 19:11:10.297313 6303 ts_tablet_manager.cc:1428] T b2ae87b5f3d249f48c61b49b5f5fbc77 P 02c4a487189f44b69b7f49f33a486807: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
W20260811 19:11:10.454957 5995 tablet.cc:2369] T b2ae87b5f3d249f48c61b49b5f5fbc77 P df6626ebdf524b78a3551525563ac21d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:11:10.524554 6316 raft_consensus.cc:482] T b2ae87b5f3d249f48c61b49b5f5fbc77 P 2e855b00e2984a8f986aaf94518ab798 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:11:10.524852 6316 raft_consensus.cc:504] T b2ae87b5f3d249f48c61b49b5f5fbc77 P 2e855b00e2984a8f986aaf94518ab798 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c4a487189f44b69b7f49f33a486807" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 42915 } } peers { permanent_uuid: "df6626ebdf524b78a3551525563ac21d" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41691 } } peers { permanent_uuid: "2e855b00e2984a8f986aaf94518ab798" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37461 } }
I20260811 19:11:10.527020 6316 leader_election.cc:288] T b2ae87b5f3d249f48c61b49b5f5fbc77 P 2e855b00e2984a8f986aaf94518ab798 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02c4a487189f44b69b7f49f33a486807 (127.3.61.129:42915), df6626ebdf524b78a3551525563ac21d (127.3.61.130:41691)
I20260811 19:11:10.540927 6293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:10.531793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:43101 (local address 127.3.61.129:42915)
0811 19:11:10.532152 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:10.532159 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:10.532758 (+ 599us) server_negotiation.cc:407] Connection header received
0811 19:11:10.533110 (+ 352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:10.533114 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:10.533185 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:10.533271 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:11:10.533983 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:10.535555 (+ 1572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:10.538531 (+ 2976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:10.539544 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:10.539792 (+ 248us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:10.539900 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:10.540329 (+ 429us) server_negotiation.cc:299] Negotiation successful
0811 19:11:10.540544 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205}
I20260811 19:11:10.541095 6298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:10.531533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:51803 (local address 127.3.61.130:41691)
0811 19:11:10.531725 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:10.531731 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:10.533281 (+ 1550us) server_negotiation.cc:407] Connection header received
0811 19:11:10.533540 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:10.533546 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:10.533675 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:10.533771 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:11:10.534509 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:10.535897 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:10.538599 (+ 2702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:10.539856 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:10.540065 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:10.540150 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:10.540598 (+ 448us) server_negotiation.cc:299] Negotiation successful
0811 19:11:10.540799 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 19:11:10.540951 6356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:10.527316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:42915 (local address 127.3.61.131:43101)
0811 19:11:10.532620 (+ 5304us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:10.532644 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:10.532906 (+ 262us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:10.533368 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:10.533373 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:10.533399 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:11:10.533738 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:10.533754 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:10.535804 (+ 2050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:10.535819 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:10.538246 (+ 2427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:10.538266 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:10.539793 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:10.539800 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:10.540025 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:10.540072 (+ 47us) client_negotiation.cc:769] Sending connection context
0811 19:11:10.540239 (+ 167us) client_negotiation.cc:240] Negotiation successful
0811 19:11:10.540446 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5168,"mutex_wait_us":84,"thread_start_us":5492,"threads_started":1}
I20260811 19:11:10.542201 6355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:10.527688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.130:41691 (local address 127.3.61.131:51803)
0811 19:11:10.533150 (+ 5462us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:10.533171 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:10.533319 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:10.533841 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:10.533846 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:10.533875 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:11:10.534236 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:10.534250 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:10.536150 (+ 1900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:10.536174 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:10.538308 (+ 2134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:10.538336 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:10.540148 (+ 1812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:10.540155 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:10.540361 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:10.540393 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 19:11:10.541445 (+ 1052us) client_negotiation.cc:240] Negotiation successful
0811 19:11:10.541758 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5337,"mutex_wait_us":10,"thread_start_us":4627,"threads_started":1}
I20260811 19:11:10.542716 5596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2ae87b5f3d249f48c61b49b5f5fbc77" candidate_uuid: "2e855b00e2984a8f986aaf94518ab798" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c4a487189f44b69b7f49f33a486807" is_pre_election: true
I20260811 19:11:10.543038 5891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2ae87b5f3d249f48c61b49b5f5fbc77" candidate_uuid: "2e855b00e2984a8f986aaf94518ab798" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df6626ebdf524b78a3551525563ac21d" is_pre_election: true
I20260811 19:11:10.543220 5596 raft_consensus.cc:2454] T b2ae87b5f3d249f48c61b49b5f5fbc77 P 02c4a487189f44b69b7f49f33a486807 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e855b00e2984a8f986aaf94518ab798 in term 0.
I20260811 19:11:10.543437 5891 raft_consensus.cc:2454] T b2ae87b5f3d249f48c61b49b5f5fbc77 P df6626ebdf524b78a3551525563ac21d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e855b00e2984a8f986aaf94518ab798 in term 0.
I20260811 19:11:10.544066 6030 leader_election.cc:302] T b2ae87b5f3d249f48c61b49b5f5fbc77 P 2e855b00e2984a8f986aaf94518ab798 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e855b00e2984a8f986aaf94518ab798, df6626ebdf524b78a3551525563ac21d; no voters:
I20260811 19:11:10.544565 6316 raft_consensus.cc:2781] T b2ae87b5f3d249f48c61b49b5f5fbc77 P 2e855b00e2984a8f986aaf94518ab798 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:11:10.544719 6316 raft_consensus.cc:482] T b2ae87b5f3d249f48c61b49b5f5fbc77 P 2e855b00e2984a8f986aaf94518ab798 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:11:10.544821 6316 raft_consensus.cc:3037] T b2ae87b5f3d249f48c61b49b5f5fbc77 P 2e855b00e2984a8f986aaf94518ab798 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:11:10.548316 6316 raft_consensus.cc:504] T b2ae87b5f3d249f48c61b49b5f5fbc77 P 2e855b00e2984a8f986aaf94518ab798 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c4a487189f44b69b7f49f33a486807" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 42915 } } peers { permanent_uuid: "df6626ebdf524b78a3551525563ac21d" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41691 } } peers { permanent_uuid: "2e855b00e2984a8f986aaf94518ab798" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37461 } }
I20260811 19:11:10.549186 6316 leader_election.cc:288] T b2ae87b5f3d249f48c61b49b5f5fbc77 P 2e855b00e2984a8f986aaf94518ab798 [CANDIDATE]: Term 1 election: Requested vote from peers 02c4a487189f44b69b7f49f33a486807 (127.3.61.129:42915), df6626ebdf524b78a3551525563ac21d (127.3.61.130:41691)
I20260811 19:11:10.549887 5891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2ae87b5f3d249f48c61b49b5f5fbc77" candidate_uuid: "2e855b00e2984a8f986aaf94518ab798" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df6626ebdf524b78a3551525563ac21d"
I20260811 19:11:10.550154 5891 raft_consensus.cc:3037] T b2ae87b5f3d249f48c61b49b5f5fbc77 P df6626ebdf524b78a3551525563ac21d [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:11:10.553365 5891 raft_consensus.cc:2454] T b2ae87b5f3d249f48c61b49b5f5fbc77 P df6626ebdf524b78a3551525563ac21d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e855b00e2984a8f986aaf94518ab798 in term 1.
I20260811 19:11:10.553422 5596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2ae87b5f3d249f48c61b49b5f5fbc77" candidate_uuid: "2e855b00e2984a8f986aaf94518ab798" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c4a487189f44b69b7f49f33a486807"
I20260811 19:11:10.553740 5596 raft_consensus.cc:3037] T b2ae87b5f3d249f48c61b49b5f5fbc77 P 02c4a487189f44b69b7f49f33a486807 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:11:10.553985 6030 leader_election.cc:302] T b2ae87b5f3d249f48c61b49b5f5fbc77 P 2e855b00e2984a8f986aaf94518ab798 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e855b00e2984a8f986aaf94518ab798, df6626ebdf524b78a3551525563ac21d; no voters:
I20260811 19:11:10.554359 6316 raft_consensus.cc:2781] T b2ae87b5f3d249f48c61b49b5f5fbc77 P 2e855b00e2984a8f986aaf94518ab798 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:11:10.557617 5596 raft_consensus.cc:2454] T b2ae87b5f3d249f48c61b49b5f5fbc77 P 02c4a487189f44b69b7f49f33a486807 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e855b00e2984a8f986aaf94518ab798 in term 1.
I20260811 19:11:10.558116 6316 raft_consensus.cc:686] T b2ae87b5f3d249f48c61b49b5f5fbc77 P 2e855b00e2984a8f986aaf94518ab798 [term 1 LEADER]: Becoming Leader. State: Replica: 2e855b00e2984a8f986aaf94518ab798, State: Running, Role: LEADER
I20260811 19:11:10.558627 6316 consensus_queue.cc:237] T b2ae87b5f3d249f48c61b49b5f5fbc77 P 2e855b00e2984a8f986aaf94518ab798 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c4a487189f44b69b7f49f33a486807" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 42915 } } peers { permanent_uuid: "df6626ebdf524b78a3551525563ac21d" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41691 } } peers { permanent_uuid: "2e855b00e2984a8f986aaf94518ab798" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37461 } }
I20260811 19:11:10.562755 5298 catalog_manager.cc:5495] T b2ae87b5f3d249f48c61b49b5f5fbc77 P 2e855b00e2984a8f986aaf94518ab798 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2e855b00e2984a8f986aaf94518ab798 (127.3.61.131). New cstate: current_term: 1 leader_uuid: "2e855b00e2984a8f986aaf94518ab798" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c4a487189f44b69b7f49f33a486807" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 42915 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df6626ebdf524b78a3551525563ac21d" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2e855b00e2984a8f986aaf94518ab798" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37461 } health_report { overall_health: HEALTHY } } }
Aug 11 19:11:10 dist-test-slave-t7g5 krb5kdc[5220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475467, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.190@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:11:10.627089 6370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:10.600087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:42953 (local address 127.0.0.1:53510)
0811 19:11:10.600841 (+ 754us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:10.600874 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:10.601146 (+ 272us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:10.601538 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:10.601543 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:10.602598 (+ 1055us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:11:10.602917 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:10.602931 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:10.604293 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:10.604299 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:10.606156 (+ 1857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:10.606194 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:10.607378 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:10.607390 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:10.607638 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:10.615384 (+ 7746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:11:10.615417 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:11:10.618591 (+ 3174us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:11:10.623735 (+ 5144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:10.623754 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:10.623763 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:10.624393 (+ 630us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:10.624911 (+ 518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:10.624916 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:10.624919 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:10.625099 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:10.625908 (+ 809us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:11:10.625919 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:11:10.626135 (+ 216us) client_negotiation.cc:769] Sending connection context
0811 19:11:10.626305 (+ 170us) client_negotiation.cc:240] Negotiation successful
0811 19:11:10.626630 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":391,"threads_started":1}
I20260811 19:11:10.627658 6368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:10.600138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53510 (local address 127.3.61.190:42953)
0811 19:11:10.601239 (+ 1101us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:10.601243 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:10.601264 (+ 21us) server_negotiation.cc:407] Connection header received
0811 19:11:10.601317 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:10.601320 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:10.601375 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:10.601465 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:10.603129 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:10.604118 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:10.606489 (+ 2371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:10.607053 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:10.607230 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:10.618934 (+ 11704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:10.618943 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:10.618948 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:10.618996 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:10.623518 (+ 4522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:10.624568 (+ 1050us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:10.624574 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:10.624577 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:10.624758 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:10.625249 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:10.625254 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:10.625257 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:10.625736 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:10.625929 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:10.626458 (+ 529us) server_negotiation.cc:299] Negotiation successful
0811 19:11:10.626836 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":100,"server-negotiator.queue_time_us":994,"thread_start_us":859,"threads_started":1}
I20260811 19:11:10.634943 5298 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:53510:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 19:11:10.636395 5298 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:11:10.645785 6124 tablet_service.cc:1467] Processing CreateTablet for tablet 46404a896f6d4da489dd192c45d1406b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=080ec23394e44001b1ac890ceaa9d339]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 19:11:10.645776 5546 tablet_service.cc:1467] Processing CreateTablet for tablet 46404a896f6d4da489dd192c45d1406b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=080ec23394e44001b1ac890ceaa9d339]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 19:11:10.646461 5546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46404a896f6d4da489dd192c45d1406b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:11:10.646484 6124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46404a896f6d4da489dd192c45d1406b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:11:10.650626 5841 tablet_service.cc:1467] Processing CreateTablet for tablet 46404a896f6d4da489dd192c45d1406b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=080ec23394e44001b1ac890ceaa9d339]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 19:11:10.651257 5841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46404a896f6d4da489dd192c45d1406b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:11:10.657243 6303 tablet_bootstrap.cc:492] T 46404a896f6d4da489dd192c45d1406b P 02c4a487189f44b69b7f49f33a486807: Bootstrap starting.
I20260811 19:11:10.658459 6309 tablet_bootstrap.cc:492] T 46404a896f6d4da489dd192c45d1406b P 2e855b00e2984a8f986aaf94518ab798: Bootstrap starting.
I20260811 19:11:10.659407 6303 tablet_bootstrap.cc:654] T 46404a896f6d4da489dd192c45d1406b P 02c4a487189f44b69b7f49f33a486807: Neither blocks nor log segments found. Creating new log.
I20260811 19:11:10.659495 6310 tablet_bootstrap.cc:492] T 46404a896f6d4da489dd192c45d1406b P df6626ebdf524b78a3551525563ac21d: Bootstrap starting.
I20260811 19:11:10.660234 6309 tablet_bootstrap.cc:654] T 46404a896f6d4da489dd192c45d1406b P 2e855b00e2984a8f986aaf94518ab798: Neither blocks nor log segments found. Creating new log.
I20260811 19:11:10.661163 6310 tablet_bootstrap.cc:654] T 46404a896f6d4da489dd192c45d1406b P df6626ebdf524b78a3551525563ac21d: Neither blocks nor log segments found. Creating new log.
I20260811 19:11:10.672369 6303 tablet_bootstrap.cc:492] T 46404a896f6d4da489dd192c45d1406b P 02c4a487189f44b69b7f49f33a486807: No bootstrap required, opened a new log
I20260811 19:11:10.672640 6310 tablet_bootstrap.cc:492] T 46404a896f6d4da489dd192c45d1406b P df6626ebdf524b78a3551525563ac21d: No bootstrap required, opened a new log
I20260811 19:11:10.672665 6303 ts_tablet_manager.cc:1397] T 46404a896f6d4da489dd192c45d1406b P 02c4a487189f44b69b7f49f33a486807: Time spent bootstrapping tablet: real 0.015s user 0.010s sys 0.000s
I20260811 19:11:10.672863 6310 ts_tablet_manager.cc:1397] T 46404a896f6d4da489dd192c45d1406b P df6626ebdf524b78a3551525563ac21d: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.004s
I20260811 19:11:10.673535 6303 raft_consensus.cc:348] T 46404a896f6d4da489dd192c45d1406b P 02c4a487189f44b69b7f49f33a486807 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c4a487189f44b69b7f49f33a486807" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 42915 } } peers { permanent_uuid: "df6626ebdf524b78a3551525563ac21d" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41691 } } peers { permanent_uuid: "2e855b00e2984a8f986aaf94518ab798" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37461 } }
I20260811 19:11:10.673602 6310 raft_consensus.cc:348] T 46404a896f6d4da489dd192c45d1406b P df6626ebdf524b78a3551525563ac21d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c4a487189f44b69b7f49f33a486807" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 42915 } } peers { permanent_uuid: "df6626ebdf524b78a3551525563ac21d" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41691 } } peers { permanent_uuid: "2e855b00e2984a8f986aaf94518ab798" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37461 } }
I20260811 19:11:10.673906 6303 raft_consensus.cc:374] T 46404a896f6d4da489dd192c45d1406b P 02c4a487189f44b69b7f49f33a486807 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:11:10.673933 6310 raft_consensus.cc:374] T 46404a896f6d4da489dd192c45d1406b P df6626ebdf524b78a3551525563ac21d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:11:10.674009 6310 raft_consensus.cc:729] T 46404a896f6d4da489dd192c45d1406b P df6626ebdf524b78a3551525563ac21d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df6626ebdf524b78a3551525563ac21d, State: Initialized, Role: FOLLOWER
I20260811 19:11:10.674042 6303 raft_consensus.cc:729] T 46404a896f6d4da489dd192c45d1406b P 02c4a487189f44b69b7f49f33a486807 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02c4a487189f44b69b7f49f33a486807, State: Initialized, Role: FOLLOWER
I20260811 19:11:10.674110 6309 tablet_bootstrap.cc:492] T 46404a896f6d4da489dd192c45d1406b P 2e855b00e2984a8f986aaf94518ab798: No bootstrap required, opened a new log
I20260811 19:11:10.674371 6309 ts_tablet_manager.cc:1397] T 46404a896f6d4da489dd192c45d1406b P 2e855b00e2984a8f986aaf94518ab798: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.008s
I20260811 19:11:10.674345 6310 consensus_queue.cc:260] T 46404a896f6d4da489dd192c45d1406b P df6626ebdf524b78a3551525563ac21d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c4a487189f44b69b7f49f33a486807" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 42915 } } peers { permanent_uuid: "df6626ebdf524b78a3551525563ac21d" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41691 } } peers { permanent_uuid: "2e855b00e2984a8f986aaf94518ab798" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37461 } }
I20260811 19:11:10.674394 6303 consensus_queue.cc:260] T 46404a896f6d4da489dd192c45d1406b P 02c4a487189f44b69b7f49f33a486807 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c4a487189f44b69b7f49f33a486807" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 42915 } } peers { permanent_uuid: "df6626ebdf524b78a3551525563ac21d" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41691 } } peers { permanent_uuid: "2e855b00e2984a8f986aaf94518ab798" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37461 } }
I20260811 19:11:10.675740 6303 ts_tablet_manager.cc:1428] T 46404a896f6d4da489dd192c45d1406b P 02c4a487189f44b69b7f49f33a486807: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 19:11:10.675027 6315 tablet_replica.cc:439] T 46404a896f6d4da489dd192c45d1406b P df6626ebdf524b78a3551525563ac21d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c4a487189f44b69b7f49f33a486807" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 42915 } } peers { permanent_uuid: "df6626ebdf524b78a3551525563ac21d" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41691 } } peers { permanent_uuid: "2e855b00e2984a8f986aaf94518ab798" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37461 } } }
I20260811 19:11:10.677093 6315 tablet_replica.cc:442] T 46404a896f6d4da489dd192c45d1406b P df6626ebdf524b78a3551525563ac21d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 19:11:10.676985 6319 tablet_replica.cc:439] T 46404a896f6d4da489dd192c45d1406b P 02c4a487189f44b69b7f49f33a486807: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c4a487189f44b69b7f49f33a486807" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 42915 } } peers { permanent_uuid: "df6626ebdf524b78a3551525563ac21d" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41691 } } peers { permanent_uuid: "2e855b00e2984a8f986aaf94518ab798" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37461 } } }
I20260811 19:11:10.677327 6319 tablet_replica.cc:442] T 46404a896f6d4da489dd192c45d1406b P 02c4a487189f44b69b7f49f33a486807: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 19:11:10.677341 6310 ts_tablet_manager.cc:1428] T 46404a896f6d4da489dd192c45d1406b P df6626ebdf524b78a3551525563ac21d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 19:11:10.676745 6309 raft_consensus.cc:348] T 46404a896f6d4da489dd192c45d1406b P 2e855b00e2984a8f986aaf94518ab798 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c4a487189f44b69b7f49f33a486807" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 42915 } } peers { permanent_uuid: "df6626ebdf524b78a3551525563ac21d" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41691 } } peers { permanent_uuid: "2e855b00e2984a8f986aaf94518ab798" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37461 } }
I20260811 19:11:10.678403 6309 raft_consensus.cc:374] T 46404a896f6d4da489dd192c45d1406b P 2e855b00e2984a8f986aaf94518ab798 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:11:10.678545 6309 raft_consensus.cc:729] T 46404a896f6d4da489dd192c45d1406b P 2e855b00e2984a8f986aaf94518ab798 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e855b00e2984a8f986aaf94518ab798, State: Initialized, Role: FOLLOWER
I20260811 19:11:10.679184 6309 consensus_queue.cc:260] T 46404a896f6d4da489dd192c45d1406b P 2e855b00e2984a8f986aaf94518ab798 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c4a487189f44b69b7f49f33a486807" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 42915 } } peers { permanent_uuid: "df6626ebdf524b78a3551525563ac21d" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41691 } } peers { permanent_uuid: "2e855b00e2984a8f986aaf94518ab798" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37461 } }
I20260811 19:11:10.679898 6316 tablet_replica.cc:439] T 46404a896f6d4da489dd192c45d1406b P 2e855b00e2984a8f986aaf94518ab798: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c4a487189f44b69b7f49f33a486807" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 42915 } } peers { permanent_uuid: "df6626ebdf524b78a3551525563ac21d" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41691 } } peers { permanent_uuid: "2e855b00e2984a8f986aaf94518ab798" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37461 } } }
I20260811 19:11:10.680205 6316 tablet_replica.cc:442] T 46404a896f6d4da489dd192c45d1406b P 2e855b00e2984a8f986aaf94518ab798: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 19:11:10.681233 6309 ts_tablet_manager.cc:1428] T 46404a896f6d4da489dd192c45d1406b P 2e855b00e2984a8f986aaf94518ab798: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20260811 19:11:10.760548 6319 raft_consensus.cc:482] T 46404a896f6d4da489dd192c45d1406b P 02c4a487189f44b69b7f49f33a486807 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:11:10.760793 6319 raft_consensus.cc:504] T 46404a896f6d4da489dd192c45d1406b P 02c4a487189f44b69b7f49f33a486807 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c4a487189f44b69b7f49f33a486807" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 42915 } } peers { permanent_uuid: "df6626ebdf524b78a3551525563ac21d" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41691 } } peers { permanent_uuid: "2e855b00e2984a8f986aaf94518ab798" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37461 } }
I20260811 19:11:10.761446 6319 leader_election.cc:288] T 46404a896f6d4da489dd192c45d1406b P 02c4a487189f44b69b7f49f33a486807 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers df6626ebdf524b78a3551525563ac21d (127.3.61.130:41691), 2e855b00e2984a8f986aaf94518ab798 (127.3.61.131:37461)
I20260811 19:11:10.762199 6175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46404a896f6d4da489dd192c45d1406b" candidate_uuid: "02c4a487189f44b69b7f49f33a486807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e855b00e2984a8f986aaf94518ab798" is_pre_election: true
I20260811 19:11:10.762183 5891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46404a896f6d4da489dd192c45d1406b" candidate_uuid: "02c4a487189f44b69b7f49f33a486807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df6626ebdf524b78a3551525563ac21d" is_pre_election: true
I20260811 19:11:10.762460 6175 raft_consensus.cc:2454] T 46404a896f6d4da489dd192c45d1406b P 2e855b00e2984a8f986aaf94518ab798 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02c4a487189f44b69b7f49f33a486807 in term 0.
I20260811 19:11:10.762471 5891 raft_consensus.cc:2454] T 46404a896f6d4da489dd192c45d1406b P df6626ebdf524b78a3551525563ac21d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02c4a487189f44b69b7f49f33a486807 in term 0.
I20260811 19:11:10.762883 5461 leader_election.cc:302] T 46404a896f6d4da489dd192c45d1406b P 02c4a487189f44b69b7f49f33a486807 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c4a487189f44b69b7f49f33a486807, df6626ebdf524b78a3551525563ac21d; no voters:
I20260811 19:11:10.763193 6319 raft_consensus.cc:2781] T 46404a896f6d4da489dd192c45d1406b P 02c4a487189f44b69b7f49f33a486807 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:11:10.763293 6319 raft_consensus.cc:482] T 46404a896f6d4da489dd192c45d1406b P 02c4a487189f44b69b7f49f33a486807 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:11:10.763451 6319 raft_consensus.cc:3037] T 46404a896f6d4da489dd192c45d1406b P 02c4a487189f44b69b7f49f33a486807 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:11:10.766119 6319 raft_consensus.cc:504] T 46404a896f6d4da489dd192c45d1406b P 02c4a487189f44b69b7f49f33a486807 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c4a487189f44b69b7f49f33a486807" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 42915 } } peers { permanent_uuid: "df6626ebdf524b78a3551525563ac21d" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41691 } } peers { permanent_uuid: "2e855b00e2984a8f986aaf94518ab798" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37461 } }
I20260811 19:11:10.766988 6319 leader_election.cc:288] T 46404a896f6d4da489dd192c45d1406b P 02c4a487189f44b69b7f49f33a486807 [CANDIDATE]: Term 1 election: Requested vote from peers df6626ebdf524b78a3551525563ac21d (127.3.61.130:41691), 2e855b00e2984a8f986aaf94518ab798 (127.3.61.131:37461)
I20260811 19:11:10.767619 5891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46404a896f6d4da489dd192c45d1406b" candidate_uuid: "02c4a487189f44b69b7f49f33a486807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df6626ebdf524b78a3551525563ac21d"
I20260811 19:11:10.767904 5891 raft_consensus.cc:3037] T 46404a896f6d4da489dd192c45d1406b P df6626ebdf524b78a3551525563ac21d [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:11:10.767923 6175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46404a896f6d4da489dd192c45d1406b" candidate_uuid: "02c4a487189f44b69b7f49f33a486807" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e855b00e2984a8f986aaf94518ab798"
I20260811 19:11:10.768302 6175 raft_consensus.cc:3037] T 46404a896f6d4da489dd192c45d1406b P 2e855b00e2984a8f986aaf94518ab798 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:11:10.770959 5891 raft_consensus.cc:2454] T 46404a896f6d4da489dd192c45d1406b P df6626ebdf524b78a3551525563ac21d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02c4a487189f44b69b7f49f33a486807 in term 1.
I20260811 19:11:10.771478 5461 leader_election.cc:302] T 46404a896f6d4da489dd192c45d1406b P 02c4a487189f44b69b7f49f33a486807 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c4a487189f44b69b7f49f33a486807, df6626ebdf524b78a3551525563ac21d; no voters:
I20260811 19:11:10.771905 6319 raft_consensus.cc:2781] T 46404a896f6d4da489dd192c45d1406b P 02c4a487189f44b69b7f49f33a486807 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:11:10.772478 6175 raft_consensus.cc:2454] T 46404a896f6d4da489dd192c45d1406b P 2e855b00e2984a8f986aaf94518ab798 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02c4a487189f44b69b7f49f33a486807 in term 1.
I20260811 19:11:10.772536 6319 raft_consensus.cc:686] T 46404a896f6d4da489dd192c45d1406b P 02c4a487189f44b69b7f49f33a486807 [term 1 LEADER]: Becoming Leader. State: Replica: 02c4a487189f44b69b7f49f33a486807, State: Running, Role: LEADER
I20260811 19:11:10.772934 6319 consensus_queue.cc:237] T 46404a896f6d4da489dd192c45d1406b P 02c4a487189f44b69b7f49f33a486807 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c4a487189f44b69b7f49f33a486807" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 42915 } } peers { permanent_uuid: "df6626ebdf524b78a3551525563ac21d" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41691 } } peers { permanent_uuid: "2e855b00e2984a8f986aaf94518ab798" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37461 } }
I20260811 19:11:10.775074 6377 tablet_replica.cc:439] T 46404a896f6d4da489dd192c45d1406b P 02c4a487189f44b69b7f49f33a486807: TxnStatusTablet state changed. Reason: New leader 02c4a487189f44b69b7f49f33a486807. Latest consensus state: current_term: 1 leader_uuid: "02c4a487189f44b69b7f49f33a486807" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c4a487189f44b69b7f49f33a486807" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 42915 } } peers { permanent_uuid: "df6626ebdf524b78a3551525563ac21d" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41691 } } peers { permanent_uuid: "2e855b00e2984a8f986aaf94518ab798" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37461 } } }
I20260811 19:11:10.775360 6377 tablet_replica.cc:442] T 46404a896f6d4da489dd192c45d1406b P 02c4a487189f44b69b7f49f33a486807: This TxnStatusTablet replica's current role is: LEADER
I20260811 19:11:10.777454 6381 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 19:11:10.778088 6381 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 19:11:10.777935 5298 catalog_manager.cc:5495] T 46404a896f6d4da489dd192c45d1406b P 02c4a487189f44b69b7f49f33a486807 reported cstate change: term changed from 0 to 1, leader changed from <none> to 02c4a487189f44b69b7f49f33a486807 (127.3.61.129). New cstate: current_term: 1 leader_uuid: "02c4a487189f44b69b7f49f33a486807" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c4a487189f44b69b7f49f33a486807" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 42915 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "df6626ebdf524b78a3551525563ac21d" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2e855b00e2984a8f986aaf94518ab798" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37461 } health_report { overall_health: UNKNOWN } } }
I20260811 19:11:10.806440 6370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:10.798316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:42915 (local address 127.0.0.1:56770)
0811 19:11:10.798578 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:10.798597 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:10.798739 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:10.799257 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:10.799261 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:10.799272 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 19:11:10.799557 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:10.799568 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:10.801167 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:10.801173 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:10.802955 (+ 1782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:10.802979 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:10.804581 (+ 1602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:10.804591 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:10.805014 (+ 423us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:10.805116 (+ 102us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 19:11:10.805771 (+ 655us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 19:11:10.805807 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 19:11:10.805897 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 19:11:10.806100 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260811 19:11:10.806531 6293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:10.798313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56770 (local address 127.3.61.129:42915)
0811 19:11:10.798711 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:10.798717 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:10.798742 (+ 25us) server_negotiation.cc:407] Connection header received
0811 19:11:10.798929 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:10.798936 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:10.799081 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:10.799204 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:11:10.799717 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:10.800864 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:10.803313 (+ 2449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:10.804245 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:10.804413 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:10.805376 (+ 963us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:11:10.805594 (+ 218us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:11:10.805730 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:10.805947 (+ 217us) server_negotiation.cc:299] Negotiation successful
0811 19:11:10.806040 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264}
I20260811 19:11:10.814388 6175 raft_consensus.cc:1261] T 46404a896f6d4da489dd192c45d1406b P 2e855b00e2984a8f986aaf94518ab798 [term 1 FOLLOWER]: Refusing update from remote peer 02c4a487189f44b69b7f49f33a486807: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:11:10.814735 5891 raft_consensus.cc:1261] T 46404a896f6d4da489dd192c45d1406b P df6626ebdf524b78a3551525563ac21d [term 1 FOLLOWER]: Refusing update from remote peer 02c4a487189f44b69b7f49f33a486807: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:11:10.815172 6377 consensus_queue.cc:1035] T 46404a896f6d4da489dd192c45d1406b P 02c4a487189f44b69b7f49f33a486807 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e855b00e2984a8f986aaf94518ab798" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:11:10.815888 6319 consensus_queue.cc:1035] T 46404a896f6d4da489dd192c45d1406b P 02c4a487189f44b69b7f49f33a486807 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df6626ebdf524b78a3551525563ac21d" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:11:10.835319 6315 tablet_replica.cc:439] T 46404a896f6d4da489dd192c45d1406b P df6626ebdf524b78a3551525563ac21d: TxnStatusTablet state changed. Reason: New leader 02c4a487189f44b69b7f49f33a486807. Latest consensus state: current_term: 1 leader_uuid: "02c4a487189f44b69b7f49f33a486807" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c4a487189f44b69b7f49f33a486807" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 42915 } } peers { permanent_uuid: "df6626ebdf524b78a3551525563ac21d" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41691 } } peers { permanent_uuid: "2e855b00e2984a8f986aaf94518ab798" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37461 } } }
I20260811 19:11:10.835855 6315 tablet_replica.cc:442] T 46404a896f6d4da489dd192c45d1406b P df6626ebdf524b78a3551525563ac21d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 19:11:10.838528 6316 tablet_replica.cc:439] T 46404a896f6d4da489dd192c45d1406b P 2e855b00e2984a8f986aaf94518ab798: TxnStatusTablet state changed. Reason: New leader 02c4a487189f44b69b7f49f33a486807. Latest consensus state: current_term: 1 leader_uuid: "02c4a487189f44b69b7f49f33a486807" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c4a487189f44b69b7f49f33a486807" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 42915 } } peers { permanent_uuid: "df6626ebdf524b78a3551525563ac21d" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41691 } } peers { permanent_uuid: "2e855b00e2984a8f986aaf94518ab798" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37461 } } }
I20260811 19:11:10.838990 6316 tablet_replica.cc:442] T 46404a896f6d4da489dd192c45d1406b P 2e855b00e2984a8f986aaf94518ab798: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 19:11:10.845728 6319 tablet_replica.cc:439] T 46404a896f6d4da489dd192c45d1406b P 02c4a487189f44b69b7f49f33a486807: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "02c4a487189f44b69b7f49f33a486807" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c4a487189f44b69b7f49f33a486807" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 42915 } } peers { permanent_uuid: "df6626ebdf524b78a3551525563ac21d" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41691 } } peers { permanent_uuid: "2e855b00e2984a8f986aaf94518ab798" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37461 } } }
I20260811 19:11:10.846285 6319 tablet_replica.cc:442] T 46404a896f6d4da489dd192c45d1406b P 02c4a487189f44b69b7f49f33a486807: This TxnStatusTablet replica's current role is: LEADER
I20260811 19:11:10.846532 6319 tablet_replica.cc:439] T 46404a896f6d4da489dd192c45d1406b P 02c4a487189f44b69b7f49f33a486807: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "02c4a487189f44b69b7f49f33a486807" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c4a487189f44b69b7f49f33a486807" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 42915 } } peers { permanent_uuid: "df6626ebdf524b78a3551525563ac21d" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41691 } } peers { permanent_uuid: "2e855b00e2984a8f986aaf94518ab798" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37461 } } }
I20260811 19:11:10.846907 6319 tablet_replica.cc:442] T 46404a896f6d4da489dd192c45d1406b P 02c4a487189f44b69b7f49f33a486807: This TxnStatusTablet replica's current role is: LEADER
I20260811 19:11:10.847141 6315 tablet_replica.cc:439] T 46404a896f6d4da489dd192c45d1406b P df6626ebdf524b78a3551525563ac21d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "02c4a487189f44b69b7f49f33a486807" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c4a487189f44b69b7f49f33a486807" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 42915 } } peers { permanent_uuid: "df6626ebdf524b78a3551525563ac21d" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41691 } } peers { permanent_uuid: "2e855b00e2984a8f986aaf94518ab798" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37461 } } }
I20260811 19:11:10.847494 6315 tablet_replica.cc:442] T 46404a896f6d4da489dd192c45d1406b P df6626ebdf524b78a3551525563ac21d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 19:11:10.858620 6316 tablet_replica.cc:439] T 46404a896f6d4da489dd192c45d1406b P 2e855b00e2984a8f986aaf94518ab798: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "02c4a487189f44b69b7f49f33a486807" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c4a487189f44b69b7f49f33a486807" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 42915 } } peers { permanent_uuid: "df6626ebdf524b78a3551525563ac21d" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41691 } } peers { permanent_uuid: "2e855b00e2984a8f986aaf94518ab798" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37461 } } }
I20260811 19:11:10.859611 6316 tablet_replica.cc:442] T 46404a896f6d4da489dd192c45d1406b P 2e855b00e2984a8f986aaf94518ab798: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 19:11:10.874886 6407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:10.864950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32880 (local address 127.3.61.131:37461)
0811 19:11:10.866394 (+ 1444us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:10.866399 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:10.866414 (+ 15us) server_negotiation.cc:407] Connection header received
0811 19:11:10.866509 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:10.866513 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:10.866689 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:10.867055 (+ 366us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:11:10.867881 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:10.869111 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:10.871197 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:10.872038 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:10.872178 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:10.872915 (+ 737us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:11:10.873922 (+ 1007us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:11:10.874080 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:10.874187 (+ 107us) server_negotiation.cc:299] Negotiation successful
0811 19:11:10.874339 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1308,"thread_start_us":1108,"threads_started":1}
I20260811 19:11:10.902848 6293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:10.891321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:54197 (local address 127.3.61.129:42915)
0811 19:11:10.891547 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:10.891552 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:10.893199 (+ 1647us) server_negotiation.cc:407] Connection header received
0811 19:11:10.893870 (+ 671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:10.893876 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:10.893983 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:10.894085 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:11:10.895188 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:10.897036 (+ 1848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:10.899958 (+ 2922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:10.900985 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:10.901137 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:10.901904 (+ 767us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:11:10.902063 (+ 159us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:11:10.902158 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:10.902338 (+ 180us) server_negotiation.cc:299] Negotiation successful
0811 19:11:10.902426 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 19:11:10.903137 6411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:10.891544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:42915 (local address 127.3.61.131:54197)
0811 19:11:10.893057 (+ 1513us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:10.893075 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:10.893689 (+ 614us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:10.894113 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:10.894118 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:10.894132 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 19:11:10.894982 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:10.894996 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:10.897205 (+ 2209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:10.897210 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:10.899769 (+ 2559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:10.899791 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:10.901295 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:10.901303 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:10.901585 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:10.901705 (+ 120us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 19:11:10.902167 (+ 462us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 19:11:10.902182 (+ 15us) client_negotiation.cc:769] Sending connection context
0811 19:11:10.902289 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 19:11:10.902460 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1225,"thread_start_us":935,"threads_started":1}
I20260811 19:11:10.917382 5891 raft_consensus.cc:1261] T b2ae87b5f3d249f48c61b49b5f5fbc77 P df6626ebdf524b78a3551525563ac21d [term 1 FOLLOWER]: Refusing update from remote peer 2e855b00e2984a8f986aaf94518ab798: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:11:10.918632 5596 raft_consensus.cc:1261] T b2ae87b5f3d249f48c61b49b5f5fbc77 P 02c4a487189f44b69b7f49f33a486807 [term 1 FOLLOWER]: Refusing update from remote peer 2e855b00e2984a8f986aaf94518ab798: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:11:10.918432 6358 consensus_queue.cc:1035] T b2ae87b5f3d249f48c61b49b5f5fbc77 P 2e855b00e2984a8f986aaf94518ab798 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df6626ebdf524b78a3551525563ac21d" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:11:10.919711 6358 consensus_queue.cc:1035] T b2ae87b5f3d249f48c61b49b5f5fbc77 P 2e855b00e2984a8f986aaf94518ab798 [LEADER]: Connected to new peer: Peer: permanent_uuid: "02c4a487189f44b69b7f49f33a486807" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 42915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:11:10.977802 6407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:10.967810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:47237 (local address 127.3.61.131:37461)
0811 19:11:10.967997 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:10.968003 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:10.969512 (+ 1509us) server_negotiation.cc:407] Connection header received
0811 19:11:10.969775 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:10.969781 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:10.969857 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:10.969928 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:11:10.971148 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:10.972620 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:10.975165 (+ 2545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:10.975977 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:10.976055 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:10.976880 (+ 825us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:11:10.977017 (+ 137us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:11:10.977068 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:10.977246 (+ 178us) server_negotiation.cc:299] Negotiation successful
0811 19:11:10.977334 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 19:11:10.978227 6418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:10.967984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.131:37461 (local address 127.3.61.129:47237)
0811 19:11:10.969403 (+ 1419us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:10.969420 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:10.969603 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:10.969958 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:10.969962 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:10.969975 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 19:11:10.970899 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:10.970915 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:10.972826 (+ 1911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:10.972830 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:10.974981 (+ 2151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:10.974998 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:10.976350 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:10.976356 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:10.976548 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:10.976657 (+ 109us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 19:11:10.977132 (+ 475us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 19:11:10.977140 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 19:11:10.977201 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 19:11:10.977341 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1117,"thread_start_us":851,"threads_started":1}
I20260811 19:11:11.045689 5298 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53498:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:11:11.046092 5298 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53498:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:11:11.050625 5298 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6d3b329daf2c495a81783be32467ea40: Sending DeleteTablet for 3 replicas of tablet b2ae87b5f3d249f48c61b49b5f5fbc77
I20260811 19:11:11.051798 5546 tablet_service.cc:1514] Processing DeleteTablet for tablet b2ae87b5f3d249f48c61b49b5f5fbc77 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:11:11 UTC) from {username='kudu', principal='kudu/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:56752
I20260811 19:11:11.052522 6124 tablet_service.cc:1514] Processing DeleteTablet for tablet b2ae87b5f3d249f48c61b49b5f5fbc77 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:11:11 UTC) from {username='kudu', principal='kudu/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:32872
I20260811 19:11:11.053121 5841 tablet_service.cc:1514] Processing DeleteTablet for tablet b2ae87b5f3d249f48c61b49b5f5fbc77 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:11:11 UTC) from {username='kudu', principal='kudu/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:39212
I20260811 19:11:11.061019 6434 tablet_replica.cc:330] T b2ae87b5f3d249f48c61b49b5f5fbc77 P df6626ebdf524b78a3551525563ac21d: stopping tablet replica
I20260811 19:11:11.062114 6434 raft_consensus.cc:2229] T b2ae87b5f3d249f48c61b49b5f5fbc77 P df6626ebdf524b78a3551525563ac21d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:11:11.063135 6434 raft_consensus.cc:2258] T b2ae87b5f3d249f48c61b49b5f5fbc77 P df6626ebdf524b78a3551525563ac21d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:11:11.065461 6434 ts_tablet_manager.cc:1905] T b2ae87b5f3d249f48c61b49b5f5fbc77 P df6626ebdf524b78a3551525563ac21d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:11:11.071430 6424 tablet_replica.cc:330] T b2ae87b5f3d249f48c61b49b5f5fbc77 P 02c4a487189f44b69b7f49f33a486807: stopping tablet replica
I20260811 19:11:11.071980 6424 raft_consensus.cc:2229] T b2ae87b5f3d249f48c61b49b5f5fbc77 P 02c4a487189f44b69b7f49f33a486807 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:11:11.072636 6424 raft_consensus.cc:2258] T b2ae87b5f3d249f48c61b49b5f5fbc77 P 02c4a487189f44b69b7f49f33a486807 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:11:11.074167 6424 ts_tablet_manager.cc:1905] T b2ae87b5f3d249f48c61b49b5f5fbc77 P 02c4a487189f44b69b7f49f33a486807: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:11:11.078544 6430 tablet_replica.cc:330] T b2ae87b5f3d249f48c61b49b5f5fbc77 P 2e855b00e2984a8f986aaf94518ab798: stopping tablet replica
I20260811 19:11:11.082262 6430 raft_consensus.cc:2229] T b2ae87b5f3d249f48c61b49b5f5fbc77 P 2e855b00e2984a8f986aaf94518ab798 [term 1 LEADER]: Raft consensus shutting down.
I20260811 19:11:11.083278 6430 raft_consensus.cc:2258] T b2ae87b5f3d249f48c61b49b5f5fbc77 P 2e855b00e2984a8f986aaf94518ab798 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:11:11.084638 6434 ts_tablet_manager.cc:1918] T b2ae87b5f3d249f48c61b49b5f5fbc77 P df6626ebdf524b78a3551525563ac21d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 19:11:11.084918 6434 log.cc:1196] T b2ae87b5f3d249f48c61b49b5f5fbc77 P df6626ebdf524b78a3551525563ac21d: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/ts-1/wal/wals/b2ae87b5f3d249f48c61b49b5f5fbc77
I20260811 19:11:11.086223 6434 ts_tablet_manager.cc:1939] T b2ae87b5f3d249f48c61b49b5f5fbc77 P df6626ebdf524b78a3551525563ac21d: Deleting consensus metadata
I20260811 19:11:11.087616 5274 catalog_manager.cc:4841] TS df6626ebdf524b78a3551525563ac21d (127.3.61.130:41691): tablet b2ae87b5f3d249f48c61b49b5f5fbc77 (table test-table [id=12fa231950d944fd8fcbdc4852e747a9]) successfully deleted
I20260811 19:11:11.088784 6430 ts_tablet_manager.cc:1905] T b2ae87b5f3d249f48c61b49b5f5fbc77 P 2e855b00e2984a8f986aaf94518ab798: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:11:11.088971 6424 ts_tablet_manager.cc:1918] T b2ae87b5f3d249f48c61b49b5f5fbc77 P 02c4a487189f44b69b7f49f33a486807: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 19:11:11.089214 6424 log.cc:1196] T b2ae87b5f3d249f48c61b49b5f5fbc77 P 02c4a487189f44b69b7f49f33a486807: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/ts-0/wal/wals/b2ae87b5f3d249f48c61b49b5f5fbc77
I20260811 19:11:11.089948 6424 ts_tablet_manager.cc:1939] T b2ae87b5f3d249f48c61b49b5f5fbc77 P 02c4a487189f44b69b7f49f33a486807: Deleting consensus metadata
I20260811 19:11:11.091313 5273 catalog_manager.cc:4841] TS 02c4a487189f44b69b7f49f33a486807 (127.3.61.129:42915): tablet b2ae87b5f3d249f48c61b49b5f5fbc77 (table test-table [id=12fa231950d944fd8fcbdc4852e747a9]) successfully deleted
I20260811 19:11:11.096196 6430 ts_tablet_manager.cc:1918] T b2ae87b5f3d249f48c61b49b5f5fbc77 P 2e855b00e2984a8f986aaf94518ab798: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 19:11:11.096369 6430 log.cc:1196] T b2ae87b5f3d249f48c61b49b5f5fbc77 P 2e855b00e2984a8f986aaf94518ab798: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786475454804547-3318-0/minicluster-data/ts-2/wal/wals/b2ae87b5f3d249f48c61b49b5f5fbc77
I20260811 19:11:11.097187 6430 ts_tablet_manager.cc:1939] T b2ae87b5f3d249f48c61b49b5f5fbc77 P 2e855b00e2984a8f986aaf94518ab798: Deleting consensus metadata
I20260811 19:11:11.098874 5273 catalog_manager.cc:4841] TS 2e855b00e2984a8f986aaf94518ab798 (127.3.61.131:37461): tablet b2ae87b5f3d249f48c61b49b5f5fbc77 (table test-table [id=12fa231950d944fd8fcbdc4852e747a9]) successfully deleted
Aug 11 19:11:11 dist-test-slave-t7g5 krb5kdc[5220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475469, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.3.61.129@KRBTEST.COM
I20260811 19:11:11.121938 6293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:11.080651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56780 (local address 127.3.61.129:42915)
0811 19:11:11.080905 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:11.080912 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:11.099896 (+ 18984us) server_negotiation.cc:407] Connection header received
0811 19:11:11.100084 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:11.100088 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:11.100163 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:11.100247 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:11.102785 (+ 2538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:11.103733 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:11.105271 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:11.106061 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:11.106197 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:11.112426 (+ 6229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:11.112456 (+ 30us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:11.112474 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:11.112558 (+ 84us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:11.117967 (+ 5409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:11.118950 (+ 983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:11.118957 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:11.118960 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:11.119259 (+ 299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:11.119771 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:11.119775 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:11.119777 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:11.120286 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:11.120740 (+ 454us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:11.121080 (+ 340us) server_negotiation.cc:299] Negotiation successful
0811 19:11:11.121408 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260811 19:11:11.122777 5693 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56780
I20260811 19:11:11.152220 6368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:11.126099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53516 (local address 127.3.61.190:42953)
0811 19:11:11.126410 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:11.126417 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:11.133545 (+ 7128us) server_negotiation.cc:407] Connection header received
0811 19:11:11.133821 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:11.133827 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:11.133905 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:11.133969 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:11.136299 (+ 2330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:11.137438 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:11.139559 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:11.140314 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:11.140455 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:11.144301 (+ 3846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:11.144307 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:11.144311 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:11.144351 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:11.148491 (+ 4140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:11.149372 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:11.149378 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:11.149382 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:11.149622 (+ 240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:11.150344 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:11.150350 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:11.150353 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:11.150682 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:11.151052 (+ 370us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:11.151436 (+ 384us) server_negotiation.cc:299] Negotiation successful
0811 19:11:11.151743 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
W20260811 19:11:11.153226 5298 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53516
I20260811 19:11:11.155561 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 5428
I20260811 19:11:11.183470 6293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:11.156899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56782 (local address 127.3.61.129:42915)
0811 19:11:11.157974 (+ 1075us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:11.157981 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:11.167464 (+ 9483us) server_negotiation.cc:407] Connection header received
0811 19:11:11.167695 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:11.167700 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:11.167781 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:11.167887 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:11.169176 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:11.170135 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:11.171372 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:11.172275 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:11.172392 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:11.174716 (+ 2324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:11.174721 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:11.174725 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:11.174760 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:11.179685 (+ 4925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:11.180560 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:11.180566 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:11.180570 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:11.180782 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:11.181217 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:11.181221 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:11.181223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:11.181531 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:11.181889 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:11.182503 (+ 614us) server_negotiation.cc:299] Negotiation successful
0811 19:11:11.182803 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":913}
W20260811 19:11:11.184118 5693 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56782
W20260811 19:11:11.184690 3318 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 5428: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 19:11:11.184870 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 5428
I20260811 19:11:11.223403 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 5708
Aug 11 19:11:11 dist-test-slave-t7g5 krb5kdc[5220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475469, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.3.61.130@KRBTEST.COM
I20260811 19:11:11.240378 6448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:11.224146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39228 (local address 127.3.61.130:41691)
0811 19:11:11.224990 (+ 844us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:11.224994 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:11.225009 (+ 15us) server_negotiation.cc:407] Connection header received
0811 19:11:11.225060 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:11.225064 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:11.225119 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:11.225214 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:11.226483 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:11.227320 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:11.228824 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:11.229344 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:11.230491 (+ 1147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:11.234571 (+ 4080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:11.234588 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:11.234607 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:11.234645 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:11.237533 (+ 2888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:11.238246 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:11.238254 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:11.238256 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:11.238395 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:11.238773 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:11.238777 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:11.238779 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:11.239171 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:11.239467 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:11.239786 (+ 319us) server_negotiation.cc:299] Negotiation successful
0811 19:11:11.240030 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":751,"thread_start_us":616,"threads_started":1}
W20260811 19:11:11.241066 5986 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39228
W20260811 19:11:11.241755 3318 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 5708: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 19:11:11.241953 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 5708
I20260811 19:11:11.278151 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 6002
Aug 11 19:11:11 dist-test-slave-t7g5 krb5kdc[5220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475469, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.3.61.131@KRBTEST.COM
I20260811 19:11:11.295855 6407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:11.278969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32888 (local address 127.3.61.131:37461)
0811 19:11:11.279193 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:11.279199 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:11.279222 (+ 23us) server_negotiation.cc:407] Connection header received
0811 19:11:11.279338 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:11.279344 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:11.279434 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:11.279553 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:11.280958 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:11.282244 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:11.283639 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:11.284314 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:11.284453 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:11.288830 (+ 4377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:11.288849 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:11.288872 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:11.288927 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:11.292898 (+ 3971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:11.293670 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:11.293681 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:11.293684 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:11.293818 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:11.294165 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:11.294170 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:11.294173 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:11.294569 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:11.294932 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:11.295224 (+ 292us) server_negotiation.cc:299] Negotiation successful
0811 19:11:11.295453 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260811 19:11:11.296600 6261 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32888
W20260811 19:11:11.297255 3318 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 6002: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 19:11:11.297401 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 6002
I20260811 19:11:11.328167 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 5242
I20260811 19:11:11.329366 5374 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:11:11.451725 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 5242
2026-08-11T19:11:11Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (5647 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:11:11 dist-test-slave-t7g5 krb5kdc[6453](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:11:11 dist-test-slave-t7g5 krb5kdc[6453](info): set up 2 sockets
Aug 11 19:11:11 dist-test-slave-t7g5 krb5kdc[6453](info): commencing operation
krb5kdc: starting...
W20260811 19:11:12.765326 3318 mini_kdc.cc:121] Time spent starting KDC: real 1.254s user 0.001s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Aug 11 19:11:12 dist-test-slave-t7g5 krb5kdc[6453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475472, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:11:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:11:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.61.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab.
Entry for principal kudu/127.3.61.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab.
I20260811 19:11:13.031924 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:33901
--webserver_interface=127.3.61.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:40157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.61.190:33901
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:11:13.415318 6475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:11:13.415650 6475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:11:13.415745 6475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:11:13.423352 6475 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:11:13.423478 6475 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 19:11:13.423518 6475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:11:13.423580 6475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:11:13.423630 6475 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:11:13.423691 6475 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:11:13.436910 6475 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:40157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.61.190:33901
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:33901
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.61.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:11:13.438761 6475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:11:13.440160 6475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:11:13.452728 6483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:13.452620 6484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:13.452580 6487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:11:13.455008 6475 server_base.cc:1034] running on GCE node
I20260811 19:11:13.455864 6475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:11:13.457357 6475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:11:13.458889 6475 hybrid_clock.cc:648] HybridClock initialized: now 1786475473458782 us; error 53 us; skew 500 ppm
I20260811 19:11:13.459439 6475 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:11:13 dist-test-slave-t7g5 krb5kdc[6453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475473, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:11:13.463527 6475 init.cc:377] Logged in from keytab as kudu/127.3.61.190@KRBTEST.COM (short username kudu)
I20260811 19:11:13.465292 6475 webserver.cc:460] Webserver started at http://127.3.61.190:34793/ using document root <none> and password file <none>
I20260811 19:11:13.466136 6475 fs_manager.cc:359] Metadata directory not provided
I20260811 19:11:13.466252 6475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:11:13.466586 6475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:11:13.469898 6475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/master-0/data/instance:
uuid: "404597c6971d4297830d78a22a801cac"
format_stamp: "Formatted at 2026-08-11 19:11:13 on dist-test-slave-t7g5"
server_key: "4419b89c43f3d1d420332f4a9d760e95"
server_key_iv: "d5fd8d940ca98deee81b8d166032b970"
server_key_version: "encryptionkey@0"
I20260811 19:11:13.470728 6475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/master-0/wal/instance:
uuid: "404597c6971d4297830d78a22a801cac"
format_stamp: "Formatted at 2026-08-11 19:11:13 on dist-test-slave-t7g5"
server_key: "4419b89c43f3d1d420332f4a9d760e95"
server_key_iv: "d5fd8d940ca98deee81b8d166032b970"
server_key_version: "encryptionkey@0"
I20260811 19:11:13.478248 6475 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260811 19:11:13.482630 6502 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:13.484344 6475 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 19:11:13.484553 6475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/master-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/master-0/wal
uuid: "404597c6971d4297830d78a22a801cac"
format_stamp: "Formatted at 2026-08-11 19:11:13 on dist-test-slave-t7g5"
server_key: "4419b89c43f3d1d420332f4a9d760e95"
server_key_iv: "d5fd8d940ca98deee81b8d166032b970"
server_key_version: "encryptionkey@0"
I20260811 19:11:13.484786 6475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:11:13.503751 6475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:11:13.508615 6475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:11:13.508911 6475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:11:13.534495 6475 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.190:33901
I20260811 19:11:13.534520 6618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.190:33901 every 8 connection(s)
I20260811 19:11:13.536023 6475 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
I20260811 19:11:13.541497 6620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:11:13.543255 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 6475
I20260811 19:11:13.543460 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/master-0/wal/instance
I20260811 19:11:13.544147 3318 external_mini_cluster.cc:1428] Setting key 6e3392b669d9fbfe0a190560b75c24bf
I20260811 19:11:13.558014 6620 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 404597c6971d4297830d78a22a801cac: Bootstrap starting.
Aug 11 19:11:13 dist-test-slave-t7g5 krb5kdc[6453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475472, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:11:13.561723 6620 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 404597c6971d4297830d78a22a801cac: Neither blocks nor log segments found. Creating new log.
I20260811 19:11:13.563103 6620 log.cc:824] T 00000000000000000000000000000000 P 404597c6971d4297830d78a22a801cac: Log is configured to *not* fsync() on all Append() calls
I20260811 19:11:13.567591 6620 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 404597c6971d4297830d78a22a801cac: No bootstrap required, opened a new log
I20260811 19:11:13.572822 6625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:13.546908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57714 (local address 127.3.61.190:33901)
0811 19:11:13.548199 (+ 1291us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:13.548215 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:13.548291 (+ 76us) server_negotiation.cc:407] Connection header received
0811 19:11:13.550486 (+ 2195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:13.550519 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:13.551373 (+ 854us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:13.552040 (+ 667us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:13.553740 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:13.555359 (+ 1619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:13.556784 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:13.557547 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:13.557770 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:13.562432 (+ 4662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:13.562444 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:13.562465 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:13.562527 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:13.567014 (+ 4487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:13.568196 (+ 1182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:13.568206 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:13.568221 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:13.568481 (+ 260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:13.568990 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:13.568995 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:13.568998 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:13.569858 (+ 860us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:13.570190 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:13.570762 (+ 572us) server_negotiation.cc:299] Negotiation successful
0811 19:11:13.571298 (+ 536us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":950,"thread_start_us":540,"threads_started":1}
I20260811 19:11:13.578831 6620 raft_consensus.cc:348] T 00000000000000000000000000000000 P 404597c6971d4297830d78a22a801cac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "404597c6971d4297830d78a22a801cac" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33901 } }
I20260811 19:11:13.579620 6620 raft_consensus.cc:374] T 00000000000000000000000000000000 P 404597c6971d4297830d78a22a801cac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:11:13.579756 6620 raft_consensus.cc:729] T 00000000000000000000000000000000 P 404597c6971d4297830d78a22a801cac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 404597c6971d4297830d78a22a801cac, State: Initialized, Role: FOLLOWER
I20260811 19:11:13.580531 6620 consensus_queue.cc:260] T 00000000000000000000000000000000 P 404597c6971d4297830d78a22a801cac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "404597c6971d4297830d78a22a801cac" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33901 } }
I20260811 19:11:13.580879 6620 raft_consensus.cc:388] T 00000000000000000000000000000000 P 404597c6971d4297830d78a22a801cac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:11:13.581012 6620 raft_consensus.cc:482] T 00000000000000000000000000000000 P 404597c6971d4297830d78a22a801cac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:11:13.581184 6620 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 404597c6971d4297830d78a22a801cac [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:11:13.584511 6620 raft_consensus.cc:504] T 00000000000000000000000000000000 P 404597c6971d4297830d78a22a801cac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "404597c6971d4297830d78a22a801cac" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33901 } }
I20260811 19:11:13.585183 6620 leader_election.cc:302] T 00000000000000000000000000000000 P 404597c6971d4297830d78a22a801cac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 404597c6971d4297830d78a22a801cac; no voters:
I20260811 19:11:13.586038 6620 leader_election.cc:288] T 00000000000000000000000000000000 P 404597c6971d4297830d78a22a801cac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:11:13.586208 6628 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 404597c6971d4297830d78a22a801cac [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:11:13.586931 6628 raft_consensus.cc:686] T 00000000000000000000000000000000 P 404597c6971d4297830d78a22a801cac [term 1 LEADER]: Becoming Leader. State: Replica: 404597c6971d4297830d78a22a801cac, State: Running, Role: LEADER
I20260811 19:11:13.587500 6628 consensus_queue.cc:237] T 00000000000000000000000000000000 P 404597c6971d4297830d78a22a801cac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "404597c6971d4297830d78a22a801cac" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33901 } }
I20260811 19:11:13.588280 6620 sys_catalog.cc:564] T 00000000000000000000000000000000 P 404597c6971d4297830d78a22a801cac [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:11:13.590860 6629 sys_catalog.cc:455] T 00000000000000000000000000000000 P 404597c6971d4297830d78a22a801cac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "404597c6971d4297830d78a22a801cac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "404597c6971d4297830d78a22a801cac" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33901 } } }
I20260811 19:11:13.590911 6631 sys_catalog.cc:455] T 00000000000000000000000000000000 P 404597c6971d4297830d78a22a801cac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 404597c6971d4297830d78a22a801cac. Latest consensus state: current_term: 1 leader_uuid: "404597c6971d4297830d78a22a801cac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "404597c6971d4297830d78a22a801cac" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33901 } } }
I20260811 19:11:13.591456 6629 sys_catalog.cc:458] T 00000000000000000000000000000000 P 404597c6971d4297830d78a22a801cac [sys.catalog]: This master's current role is: LEADER
I20260811 19:11:13.591581 6631 sys_catalog.cc:458] T 00000000000000000000000000000000 P 404597c6971d4297830d78a22a801cac [sys.catalog]: This master's current role is: LEADER
I20260811 19:11:13.595058 6640 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:11:13.600039 6640 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:11:13.610816 6640 catalog_manager.cc:1347] Generated new cluster ID: 72af0fa8bb564e229737a460d6a4fb08
I20260811 19:11:13.610975 6640 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:11:13.628082 6640 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:11:13.629532 6640 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:11:13.640372 6640 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 404597c6971d4297830d78a22a801cac: Generated new TSK 0
I20260811 19:11:13.641880 6640 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.61.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab.
Entry for principal kudu/127.3.61.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab.
I20260811 19:11:13.705323 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.129:0
--local_ip_for_outbound_sockets=127.3.61.129
--webserver_interface=127.3.61.129
--webserver_port=0
--tserver_master_addrs=127.3.61.190:33901
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab
--principal=kudu/127.3.61.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:40157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:11:14.125969 6657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:11:14.126298 6657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:11:14.126433 6657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:11:14.134421 6657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:11:14.134545 6657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:11:14.134678 6657 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 19:11:14.134740 6657 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 19:11:14.134824 6657 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.129
I20260811 19:11:14.147989 6657 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:40157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab
--principal=kudu/127.3.61.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.61.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.61.190:33901
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.129
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:11:14.149629 6657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:11:14.151095 6657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:11:14.166577 6676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:14.166520 6673 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:14.166615 6672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:11:14.167677 6657 server_base.cc:1034] running on GCE node
I20260811 19:11:14.168668 6657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:11:14.170043 6657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:11:14.171334 6657 hybrid_clock.cc:648] HybridClock initialized: now 1786475474171269 us; error 43 us; skew 500 ppm
I20260811 19:11:14.171761 6657 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:11:14 dist-test-slave-t7g5 krb5kdc[6453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475474, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:11:14.175359 6657 init.cc:377] Logged in from keytab as kudu/127.3.61.129@KRBTEST.COM (short username kudu)
I20260811 19:11:14.177139 6657 webserver.cc:460] Webserver started at http://127.3.61.129:44367/ using document root <none> and password file <none>
I20260811 19:11:14.177937 6657 fs_manager.cc:359] Metadata directory not provided
I20260811 19:11:14.178056 6657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:11:14.178356 6657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:11:14.181516 6657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-0/data/instance:
uuid: "5943e4a12ecb475db4d131b5047b8f35"
format_stamp: "Formatted at 2026-08-11 19:11:14 on dist-test-slave-t7g5"
server_key: "64477f4483567e8dd7b368f0d1c73fa8"
server_key_iv: "561eb04667cbbd7f07622b88da3676e9"
server_key_version: "encryptionkey@0"
I20260811 19:11:14.182363 6657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-0/wal/instance:
uuid: "5943e4a12ecb475db4d131b5047b8f35"
format_stamp: "Formatted at 2026-08-11 19:11:14 on dist-test-slave-t7g5"
server_key: "64477f4483567e8dd7b368f0d1c73fa8"
server_key_iv: "561eb04667cbbd7f07622b88da3676e9"
server_key_version: "encryptionkey@0"
I20260811 19:11:14.187989 6657 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 19:11:14.194941 6688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:14.197583 6657 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260811 19:11:14.197822 6657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-0/wal
uuid: "5943e4a12ecb475db4d131b5047b8f35"
format_stamp: "Formatted at 2026-08-11 19:11:14 on dist-test-slave-t7g5"
server_key: "64477f4483567e8dd7b368f0d1c73fa8"
server_key_iv: "561eb04667cbbd7f07622b88da3676e9"
server_key_version: "encryptionkey@0"
I20260811 19:11:14.198100 6657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:11:14.211134 6657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:11:14.215868 6657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:11:14.216164 6657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:11:14.219364 6657 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:11:14.219506 6657 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:14.219619 6657 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:11:14.219694 6657 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 19:11:14 dist-test-slave-t7g5 krb5kdc[6453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475474, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.129@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:11:14.244891 6702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:14.221107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:48429 (local address 127.3.61.190:33901)
0811 19:11:14.221598 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:14.221603 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:14.223061 (+ 1458us) server_negotiation.cc:407] Connection header received
0811 19:11:14.225518 (+ 2457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:14.225524 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:14.225618 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:14.225747 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:14.228289 (+ 2542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:14.229231 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:14.231739 (+ 2508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:14.232296 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:14.232731 (+ 435us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:14.237849 (+ 5118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:14.237857 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:14.237862 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:14.237914 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:14.241562 (+ 3648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:14.242461 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:14.242466 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:14.242468 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:14.242592 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:14.243063 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:14.243067 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:14.243069 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:14.243349 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:14.243494 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:14.244272 (+ 778us) server_negotiation.cc:299] Negotiation successful
0811 19:11:14.244519 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":269,"threads_started":1}
I20260811 19:11:14.246287 6704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:14.221847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33901 (local address 127.3.61.129:48429)
0811 19:11:14.222868 (+ 1021us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:14.222916 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:14.225137 (+ 2221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:14.226290 (+ 1153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:14.226304 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:14.227133 (+ 829us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:11:14.228080 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:14.228114 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:14.229392 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:14.229397 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:14.231492 (+ 2095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:14.231508 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:14.232927 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:14.232938 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:14.233330 (+ 392us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:14.234242 (+ 912us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:11:14.234273 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:11:14.237516 (+ 3243us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:11:14.241750 (+ 4234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:14.241761 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:14.241781 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:14.242321 (+ 540us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:14.242702 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:14.242708 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:14.242712 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:14.242934 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:14.243480 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:11:14.243489 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:11:14.243871 (+ 382us) client_negotiation.cc:769] Sending connection context
0811 19:11:14.244137 (+ 266us) client_negotiation.cc:240] Negotiation successful
0811 19:11:14.244707 (+ 570us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":747,"thread_start_us":518,"threads_started":1}
I20260811 19:11:14.271567 6702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:14.252696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:40885 (local address 127.3.61.190:33901)
0811 19:11:14.253153 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:14.253157 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:14.254148 (+ 991us) server_negotiation.cc:407] Connection header received
0811 19:11:14.254325 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:14.254330 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:14.254415 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:14.254517 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:14.256074 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:14.256991 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:14.260128 (+ 3137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:14.260796 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:14.260935 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:14.263540 (+ 2605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:14.263548 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:14.263554 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:14.263599 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:14.267325 (+ 3726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:14.268487 (+ 1162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:14.268493 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:14.268497 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:14.268684 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:14.269097 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:14.269102 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:14.269106 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:14.269515 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:14.269714 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:14.270989 (+ 1275us) server_negotiation.cc:299] Negotiation successful
0811 19:11:14.271229 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341}
I20260811 19:11:14.271595 6834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:14.252643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33901 (local address 127.3.61.129:40885)
0811 19:11:14.254038 (+ 1395us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:14.254054 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:14.254208 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:14.254584 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:14.254589 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:14.255349 (+ 760us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:11:14.255877 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:14.255895 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:14.257165 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:14.257171 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:14.259874 (+ 2703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:14.259894 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:14.261019 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:14.261026 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:14.261158 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:14.262159 (+ 1001us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:11:14.262183 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:11:14.263341 (+ 1158us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:11:14.267533 (+ 4192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:14.267540 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:14.267544 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:14.268247 (+ 703us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:14.268823 (+ 576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:14.268827 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:14.268829 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:14.268978 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:14.269839 (+ 861us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:11:14.269849 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:11:14.270390 (+ 541us) client_negotiation.cc:769] Sending connection context
0811 19:11:14.270595 (+ 205us) client_negotiation.cc:240] Negotiation successful
0811 19:11:14.271058 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1267,"thread_start_us":1146,"threads_started":1}
I20260811 19:11:14.274706 6657 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.129:46395
I20260811 19:11:14.274724 6920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.129:46395 every 8 connection(s)
I20260811 19:11:14.276311 6657 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
I20260811 19:11:14.277365 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 6657
I20260811 19:11:14.277590 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-0/wal/instance
I20260811 19:11:14.278239 3318 external_mini_cluster.cc:1428] Setting key 4e6d556ea97c54a7fd9942dafbed1582
I20260811 19:11:14.278440 6925 heartbeater.cc:347] Connected to a master server at 127.3.61.190:33901
I20260811 19:11:14.278683 6925 heartbeater.cc:464] Registering TS with master...
I20260811 19:11:14.279867 6925 heartbeater.cc:511] Master 127.3.61.190:33901 requested a full tablet report, sending...
I20260811 19:11:14.282423 6536 ts_manager.cc:194] Registered new tserver with Master: 5943e4a12ecb475db4d131b5047b8f35 (127.3.61.129:46395)
I20260811 19:11:14.283980 6536 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.129@KRBTEST.COM'} at 127.3.61.129:48429
WARNING: no policy specified for kudu/127.3.61.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab.
Entry for principal kudu/127.3.61.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab.
I20260811 19:11:14.330160 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.130:0
--local_ip_for_outbound_sockets=127.3.61.130
--webserver_interface=127.3.61.130
--webserver_port=0
--tserver_master_addrs=127.3.61.190:33901
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab
--principal=kudu/127.3.61.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:40157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:11:14.684171 6935 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:11:14.684496 6935 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:11:14.684607 6935 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:11:14.692173 6935 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:11:14.692309 6935 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:11:14.692399 6935 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 19:11:14.692451 6935 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 19:11:14.692524 6935 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.130
I20260811 19:11:14.702940 6935 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:40157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab
--principal=kudu/127.3.61.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.61.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.61.190:33901
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.130
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:11:14.704456 6935 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:11:14.705888 6935 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:11:14.724028 6953 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:14.724203 6946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:14.724153 6948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:11:14.725674 6935 server_base.cc:1034] running on GCE node
I20260811 19:11:14.726614 6935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:11:14.727686 6935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:11:14.728929 6935 hybrid_clock.cc:648] HybridClock initialized: now 1786475474728877 us; error 37 us; skew 500 ppm
I20260811 19:11:14.729260 6935 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:11:14 dist-test-slave-t7g5 krb5kdc[6453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475474, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:11:14.732627 6935 init.cc:377] Logged in from keytab as kudu/127.3.61.130@KRBTEST.COM (short username kudu)
I20260811 19:11:14.734484 6935 webserver.cc:460] Webserver started at http://127.3.61.130:40595/ using document root <none> and password file <none>
I20260811 19:11:14.735293 6935 fs_manager.cc:359] Metadata directory not provided
I20260811 19:11:14.735412 6935 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:11:14.735746 6935 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:11:14.738930 6935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-1/data/instance:
uuid: "5fa054465f054cdcaef05668dd6c801c"
format_stamp: "Formatted at 2026-08-11 19:11:14 on dist-test-slave-t7g5"
server_key: "f3744d35554589a19b3fd574e0a4e50c"
server_key_iv: "40b6c7dbe4eeaf283bb9ec0d7697f555"
server_key_version: "encryptionkey@0"
I20260811 19:11:14.739755 6935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-1/wal/instance:
uuid: "5fa054465f054cdcaef05668dd6c801c"
format_stamp: "Formatted at 2026-08-11 19:11:14 on dist-test-slave-t7g5"
server_key: "f3744d35554589a19b3fd574e0a4e50c"
server_key_iv: "40b6c7dbe4eeaf283bb9ec0d7697f555"
server_key_version: "encryptionkey@0"
I20260811 19:11:14.745504 6935 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 19:11:14.750207 6964 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:14.752229 6935 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 19:11:14.752617 6935 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-1/wal
uuid: "5fa054465f054cdcaef05668dd6c801c"
format_stamp: "Formatted at 2026-08-11 19:11:14 on dist-test-slave-t7g5"
server_key: "f3744d35554589a19b3fd574e0a4e50c"
server_key_iv: "40b6c7dbe4eeaf283bb9ec0d7697f555"
server_key_version: "encryptionkey@0"
I20260811 19:11:14.752910 6935 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:11:14.776907 6935 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:11:14.782615 6935 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:11:14.782917 6935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:11:14.785905 6935 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:11:14.786028 6935 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:14.786151 6935 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:11:14.786228 6935 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 19:11:14 dist-test-slave-t7g5 krb5kdc[6453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475474, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.130@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:11:14.811609 6981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:14.786721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:42887 (local address 127.3.61.190:33901)
0811 19:11:14.787239 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:14.787245 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:14.788651 (+ 1406us) server_negotiation.cc:407] Connection header received
0811 19:11:14.791189 (+ 2538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:14.791197 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:14.791306 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:14.791445 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:14.793747 (+ 2302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:14.794729 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:14.796791 (+ 2062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:14.797233 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:14.797362 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:14.803125 (+ 5763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:14.803134 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:14.803141 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:14.803194 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:14.807126 (+ 3932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:14.808307 (+ 1181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:14.808313 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:14.808317 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:14.808467 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:14.808930 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:14.808935 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:14.808938 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:14.809231 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:14.809387 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:14.810286 (+ 899us) server_negotiation.cc:299] Negotiation successful
0811 19:11:14.810922 (+ 636us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":240,"threads_started":1}
I20260811 19:11:14.812016 6983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:14.787471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33901 (local address 127.3.61.130:42887)
0811 19:11:14.788456 (+ 985us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:14.788505 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:14.790797 (+ 2292us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:14.791743 (+ 946us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:14.791752 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:14.792528 (+ 776us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:11:14.793487 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:14.793523 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:14.794892 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:14.794898 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:14.796614 (+ 1716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:14.796629 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:14.797509 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:14.797517 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:14.797790 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:14.798834 (+ 1044us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:11:14.798876 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:11:14.802767 (+ 3891us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:11:14.807398 (+ 4631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:14.807412 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:14.807440 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:14.808165 (+ 725us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:14.808590 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:14.808594 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:14.808596 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:14.808812 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:14.809370 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:11:14.809379 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:11:14.809849 (+ 470us) client_negotiation.cc:769] Sending connection context
0811 19:11:14.810179 (+ 330us) client_negotiation.cc:240] Negotiation successful
0811 19:11:14.810725 (+ 546us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":653,"thread_start_us":431,"threads_started":1}
I20260811 19:11:14.835824 6981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:14.820765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:52635 (local address 127.3.61.190:33901)
0811 19:11:14.820983 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:14.820989 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:14.822008 (+ 1019us) server_negotiation.cc:407] Connection header received
0811 19:11:14.822261 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:14.822265 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:14.822346 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:14.822468 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:14.823795 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:14.824607 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:14.826666 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:14.827223 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:14.827353 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:14.829316 (+ 1963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:14.829322 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:14.829326 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:14.829371 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:14.832645 (+ 3274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:14.833546 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:14.833550 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:14.833553 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:14.833721 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:14.834218 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:14.834223 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:14.834227 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:14.834631 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:14.834803 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:14.835322 (+ 519us) server_negotiation.cc:299] Negotiation successful
0811 19:11:14.835559 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 19:11:14.836323 7098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:14.820827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33901 (local address 127.3.61.130:52635)
0811 19:11:14.821871 (+ 1044us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:14.821895 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:14.822086 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:14.822507 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:14.822512 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:14.823187 (+ 675us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:11:14.823604 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:14.823619 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:14.824740 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:14.824746 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:14.826485 (+ 1739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:14.826509 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:14.827452 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:14.827459 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:14.827624 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:14.828223 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:11:14.828242 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:11:14.829053 (+ 811us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:11:14.832790 (+ 3737us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:14.832796 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:14.832801 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:14.833372 (+ 571us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:14.833854 (+ 482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:14.833859 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:14.833863 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:14.834064 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:14.834769 (+ 705us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:11:14.834774 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:11:14.835031 (+ 257us) client_negotiation.cc:769] Sending connection context
0811 19:11:14.835218 (+ 187us) client_negotiation.cc:240] Negotiation successful
0811 19:11:14.835734 (+ 516us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":860,"thread_start_us":623,"threads_started":1}
I20260811 19:11:14.847782 6935 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.130:38405
I20260811 19:11:14.847862 7191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.130:38405 every 8 connection(s)
I20260811 19:11:14.849474 6935 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
I20260811 19:11:14.851970 7192 heartbeater.cc:347] Connected to a master server at 127.3.61.190:33901
I20260811 19:11:14.852247 7192 heartbeater.cc:464] Registering TS with master...
I20260811 19:11:14.852964 7192 heartbeater.cc:511] Master 127.3.61.190:33901 requested a full tablet report, sending...
I20260811 19:11:14.854568 6536 ts_manager.cc:194] Registered new tserver with Master: 5fa054465f054cdcaef05668dd6c801c (127.3.61.130:38405)
I20260811 19:11:14.855640 6536 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.130@KRBTEST.COM'} at 127.3.61.130:42887
I20260811 19:11:14.857945 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 6935
I20260811 19:11:14.858170 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-1/wal/instance
I20260811 19:11:14.859031 3318 external_mini_cluster.cc:1428] Setting key d95e671f7f6fa38bb115ff5eca8ecf26
WARNING: no policy specified for kudu/127.3.61.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab.
Entry for principal kudu/127.3.61.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab.
I20260811 19:11:14.921766 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.131:0
--local_ip_for_outbound_sockets=127.3.61.131
--webserver_interface=127.3.61.131
--webserver_port=0
--tserver_master_addrs=127.3.61.190:33901
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab
--principal=kudu/127.3.61.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:40157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:11:15.285929 7201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:11:15.286266 7201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:11:15.286408 7201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 19:11:15.287422 6925 heartbeater.cc:503] Master 127.3.61.190:33901 was elected leader, sending a full tablet report...
W20260811 19:11:15.294621 7201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:11:15.294764 7201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:11:15.294873 7201 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 19:11:15.294948 7201 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 19:11:15.295024 7201 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.131
I20260811 19:11:15.305637 7201 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:40157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab
--principal=kudu/127.3.61.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.61.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.61.190:33901
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.131
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:11:15.307147 7201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:11:15.308583 7201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:11:15.323342 7212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:15.323315 7211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:15.323269 7215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:11:15.324229 7201 server_base.cc:1034] running on GCE node
I20260811 19:11:15.324900 7201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:11:15.325994 7201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:11:15.327232 7201 hybrid_clock.cc:648] HybridClock initialized: now 1786475475327147 us; error 72 us; skew 500 ppm
I20260811 19:11:15.327633 7201 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:11:15 dist-test-slave-t7g5 krb5kdc[6453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475475, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:11:15.331859 7201 init.cc:377] Logged in from keytab as kudu/127.3.61.131@KRBTEST.COM (short username kudu)
I20260811 19:11:15.333935 7201 webserver.cc:460] Webserver started at http://127.3.61.131:37929/ using document root <none> and password file <none>
I20260811 19:11:15.334726 7201 fs_manager.cc:359] Metadata directory not provided
I20260811 19:11:15.334865 7201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:11:15.335202 7201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:11:15.338585 7201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-2/data/instance:
uuid: "b60c32f9098646edaf34b986f782a645"
format_stamp: "Formatted at 2026-08-11 19:11:15 on dist-test-slave-t7g5"
server_key: "c282b79ef45e62d48d9e588c355829d4"
server_key_iv: "21db4578ac5e6be8765e619b5180f034"
server_key_version: "encryptionkey@0"
I20260811 19:11:15.339429 7201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-2/wal/instance:
uuid: "b60c32f9098646edaf34b986f782a645"
format_stamp: "Formatted at 2026-08-11 19:11:15 on dist-test-slave-t7g5"
server_key: "c282b79ef45e62d48d9e588c355829d4"
server_key_iv: "21db4578ac5e6be8765e619b5180f034"
server_key_version: "encryptionkey@0"
I20260811 19:11:15.345922 7201 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260811 19:11:15.350967 7230 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:15.353066 7201 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260811 19:11:15.353286 7201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-2/wal
uuid: "b60c32f9098646edaf34b986f782a645"
format_stamp: "Formatted at 2026-08-11 19:11:15 on dist-test-slave-t7g5"
server_key: "c282b79ef45e62d48d9e588c355829d4"
server_key_iv: "21db4578ac5e6be8765e619b5180f034"
server_key_version: "encryptionkey@0"
I20260811 19:11:15.353533 7201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:11:15.372704 7201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:11:15.377985 7201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:11:15.378285 7201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:11:15.381541 7201 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:11:15.381678 7201 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:15.381819 7201 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:11:15.381911 7201 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 19:11:15 dist-test-slave-t7g5 krb5kdc[6453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475475, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.131@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:11:15.411944 7249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:15.382127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:35095 (local address 127.3.61.190:33901)
0811 19:11:15.383334 (+ 1207us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:15.383342 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:15.384922 (+ 1580us) server_negotiation.cc:407] Connection header received
0811 19:11:15.388445 (+ 3523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:15.388466 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:15.388606 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:15.388746 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:15.391739 (+ 2993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:15.392895 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:15.395638 (+ 2743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:15.396340 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:15.396512 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:15.403437 (+ 6925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:15.403445 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:15.403451 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:15.403520 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:15.407645 (+ 4125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:15.408864 (+ 1219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:15.408868 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:15.408870 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:15.409017 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:15.409520 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:15.409525 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:15.409529 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:15.409929 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:15.410095 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:15.411381 (+ 1286us) server_negotiation.cc:299] Negotiation successful
0811 19:11:15.411630 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1018,"thread_start_us":819,"threads_started":1}
I20260811 19:11:15.414041 7252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:15.383382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33901 (local address 127.3.61.131:35095)
0811 19:11:15.384640 (+ 1258us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:15.384721 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:15.387829 (+ 3108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:15.389244 (+ 1415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:15.389260 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:15.390273 (+ 1013us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:11:15.391461 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:15.391501 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:15.393079 (+ 1578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:15.393085 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:15.395397 (+ 2312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:15.395417 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:15.396645 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:15.396656 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:15.396964 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:15.398216 (+ 1252us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:11:15.398271 (+ 55us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:11:15.403002 (+ 4731us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:11:15.407853 (+ 4851us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:15.407870 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:15.407890 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:15.408648 (+ 758us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:15.409143 (+ 495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:15.409149 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:15.409153 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:15.409390 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:15.410089 (+ 699us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:11:15.410101 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:11:15.410784 (+ 683us) client_negotiation.cc:769] Sending connection context
0811 19:11:15.411236 (+ 452us) client_negotiation.cc:240] Negotiation successful
0811 19:11:15.412020 (+ 784us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":850,"thread_start_us":512,"threads_started":1}
I20260811 19:11:15.432871 7201 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.131:39303
I20260811 19:11:15.432874 7481 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.131:39303 every 8 connection(s)
I20260811 19:11:15.434420 7201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
I20260811 19:11:15.437773 7486 heartbeater.cc:347] Connected to a master server at 127.3.61.190:33901
I20260811 19:11:15.437960 7249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:15.421242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:50589 (local address 127.3.61.190:33901)
0811 19:11:15.421460 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:15.421467 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:15.422388 (+ 921us) server_negotiation.cc:407] Connection header received
0811 19:11:15.422588 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:15.422595 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:15.422703 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:15.422810 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:15.424424 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:15.425288 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:15.427307 (+ 2019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:15.427921 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:15.428066 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:15.431100 (+ 3034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:15.431105 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:15.431109 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:15.431146 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:15.434001 (+ 2855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:15.435036 (+ 1035us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:15.435046 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:15.435050 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:15.435375 (+ 325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:15.436121 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:15.436126 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:15.436129 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:15.436578 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:15.436749 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:15.437288 (+ 539us) server_negotiation.cc:299] Negotiation successful
0811 19:11:15.437535 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 19:11:15.438104 7426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:15.421471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33901 (local address 127.3.61.131:50589)
0811 19:11:15.422252 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:15.422273 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:15.422442 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:15.422888 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:15.422894 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:15.423706 (+ 812us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:11:15.424210 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:15.424226 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:15.425513 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:15.425518 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:15.427147 (+ 1629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:15.427164 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:15.428228 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:15.428240 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:15.428460 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:15.429321 (+ 861us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:11:15.429354 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:11:15.430853 (+ 1499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:11:15.434151 (+ 3298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:15.434157 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:15.434161 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:15.434718 (+ 557us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:15.435670 (+ 952us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:15.435678 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:15.435682 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:15.435916 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:15.436739 (+ 823us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:11:15.436744 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:11:15.436959 (+ 215us) client_negotiation.cc:769] Sending connection context
0811 19:11:15.437105 (+ 146us) client_negotiation.cc:240] Negotiation successful
0811 19:11:15.437482 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"thread_start_us":480,"threads_started":1}
I20260811 19:11:15.438177 7486 heartbeater.cc:464] Registering TS with master...
I20260811 19:11:15.439224 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 7201
I20260811 19:11:15.439375 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-2/wal/instance
I20260811 19:11:15.439371 7486 heartbeater.cc:511] Master 127.3.61.190:33901 requested a full tablet report, sending...
I20260811 19:11:15.439894 3318 external_mini_cluster.cc:1428] Setting key e8a89db4de7448fea7b472a61f7203fe
I20260811 19:11:15.441805 6535 ts_manager.cc:194] Registered new tserver with Master: b60c32f9098646edaf34b986f782a645 (127.3.61.131:39303)
I20260811 19:11:15.442947 6535 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.131@KRBTEST.COM'} at 127.3.61.131:35095
I20260811 19:11:15.444146 3318 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 19:11:15 dist-test-slave-t7g5 krb5kdc[6453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475475, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 19:11:15 dist-test-slave-t7g5 krb5kdc[6453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475475, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:11:15.516147 7249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:15.498308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33540 (local address 127.3.61.190:33901)
0811 19:11:15.498713 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:15.498720 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:15.498866 (+ 146us) server_negotiation.cc:407] Connection header received
0811 19:11:15.499109 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:15.499113 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:15.499181 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:15.499270 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:15.500817 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:15.501579 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:15.502983 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:15.503519 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:15.503652 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:15.509018 (+ 5366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:15.509025 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:15.509028 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:15.509077 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:15.512689 (+ 3612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:15.513625 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:15.513629 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:15.513632 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:15.513868 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:15.514321 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:15.514325 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:15.514327 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:15.514676 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:15.514949 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:15.515496 (+ 547us) server_negotiation.cc:299] Negotiation successful
0811 19:11:15.515735 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266}
I20260811 19:11:15.521977 6536 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33540:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 19:11:15.529474 6536 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:11:15.566431 7508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:15.556899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.130:38405 (local address 127.0.0.1:35282)
0811 19:11:15.557926 (+ 1027us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:15.557941 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:15.558191 (+ 250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:15.558751 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:15.558756 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:15.558779 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:11:15.559288 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:15.559303 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:15.561136 (+ 1833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:15.561141 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:15.563644 (+ 2503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:15.563659 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:15.565299 (+ 1640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:15.565306 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:15.565533 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:15.565595 (+ 62us) client_negotiation.cc:769] Sending connection context
0811 19:11:15.565779 (+ 184us) client_negotiation.cc:240] Negotiation successful
0811 19:11:15.566005 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":917,"mutex_wait_us":246,"thread_start_us":272,"threads_started":1}
I20260811 19:11:15.567493 7515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:15.556663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35282 (local address 127.3.61.130:38405)
0811 19:11:15.557341 (+ 678us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:15.557349 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:15.558094 (+ 745us) server_negotiation.cc:407] Connection header received
0811 19:11:15.558365 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:15.558374 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:15.558576 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:15.558735 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:11:15.559467 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:15.561009 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:15.563825 (+ 2816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:15.565073 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:15.565228 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:15.565472 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:15.566971 (+ 1499us) server_negotiation.cc:299] Negotiation successful
0811 19:11:15.567183 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":511,"thread_start_us":385,"threads_started":1}
I20260811 19:11:15.567939 7512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:15.544978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.131:39303 (local address 127.0.0.1:58612)
0811 19:11:15.557676 (+ 12698us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:15.557758 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:15.558046 (+ 288us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:15.558704 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:15.558711 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:15.558740 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:11:15.559377 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:15.559391 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:15.561296 (+ 1905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:15.561301 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:15.564048 (+ 2747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:15.564151 (+ 103us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:15.567058 (+ 2907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:15.567063 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:15.567207 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:15.567233 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 19:11:15.567319 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 19:11:15.567467 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11979,"mutex_wait_us":61,"spinlock_wait_cycles":8960,"thread_start_us":11790,"threads_started":1}
I20260811 19:11:15.568625 7511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:15.545679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39448 (local address 127.3.61.129:46395)
0811 19:11:15.556252 (+ 10573us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:15.556262 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:15.558050 (+ 1788us) server_negotiation.cc:407] Connection header received
0811 19:11:15.559094 (+ 1044us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:15.559102 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:15.559355 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:15.559552 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:11:15.560418 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:15.561907 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:15.565092 (+ 3185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:15.566614 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:15.566759 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:15.566975 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:15.568248 (+ 1273us) server_negotiation.cc:299] Negotiation successful
0811 19:11:15.568408 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10377,"thread_start_us":10862,"threads_started":1}
I20260811 19:11:15.568662 7510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:15.545638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58612 (local address 127.3.61.131:39303)
0811 19:11:15.554388 (+ 8750us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:15.554401 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:15.557896 (+ 3495us) server_negotiation.cc:407] Connection header received
0811 19:11:15.558224 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:15.558230 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:15.558501 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:15.558680 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:11:15.559593 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:15.561170 (+ 1577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:15.564429 (+ 3259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:15.566569 (+ 2140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:15.566700 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:15.566918 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:15.567388 (+ 470us) server_negotiation.cc:299] Negotiation successful
0811 19:11:15.568328 (+ 940us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8506,"thread_start_us":8242,"threads_started":1}
I20260811 19:11:15.568040 7518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:15.545054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:46395 (local address 127.0.0.1:39448)
0811 19:11:15.557676 (+ 12622us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:15.557754 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:15.557990 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:15.559780 (+ 1790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:15.559784 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:15.559808 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:11:15.560194 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:15.560207 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:15.562232 (+ 2025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:15.562242 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:15.564885 (+ 2643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:15.564902 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:15.567150 (+ 2248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:15.567156 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:15.567369 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:15.567403 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 19:11:15.567497 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 19:11:15.567649 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12322,"mutex_wait_us":33,"thread_start_us":10224,"threads_started":1}
I20260811 19:11:15.571543 7068 tablet_service.cc:1467] Processing CreateTablet for tablet 90276db4edaf4be490cf3382741fe49b (DEFAULT_TABLE table=test-table [id=b82fea74a5fe4cf3a7a3121c8d363df8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:11:15.571786 7338 tablet_service.cc:1467] Processing CreateTablet for tablet 90276db4edaf4be490cf3382741fe49b (DEFAULT_TABLE table=test-table [id=b82fea74a5fe4cf3a7a3121c8d363df8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:11:15.573243 7338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90276db4edaf4be490cf3382741fe49b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:11:15.573267 7068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90276db4edaf4be490cf3382741fe49b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:11:15.574877 6791 tablet_service.cc:1467] Processing CreateTablet for tablet 90276db4edaf4be490cf3382741fe49b (DEFAULT_TABLE table=test-table [id=b82fea74a5fe4cf3a7a3121c8d363df8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:11:15.576390 6791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90276db4edaf4be490cf3382741fe49b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:11:15.585162 7519 tablet_bootstrap.cc:492] T 90276db4edaf4be490cf3382741fe49b P b60c32f9098646edaf34b986f782a645: Bootstrap starting.
I20260811 19:11:15.588800 7520 tablet_bootstrap.cc:492] T 90276db4edaf4be490cf3382741fe49b P 5fa054465f054cdcaef05668dd6c801c: Bootstrap starting.
I20260811 19:11:15.591053 7519 tablet_bootstrap.cc:654] T 90276db4edaf4be490cf3382741fe49b P b60c32f9098646edaf34b986f782a645: Neither blocks nor log segments found. Creating new log.
I20260811 19:11:15.591603 7522 tablet_bootstrap.cc:492] T 90276db4edaf4be490cf3382741fe49b P 5943e4a12ecb475db4d131b5047b8f35: Bootstrap starting.
I20260811 19:11:15.592531 7520 tablet_bootstrap.cc:654] T 90276db4edaf4be490cf3382741fe49b P 5fa054465f054cdcaef05668dd6c801c: Neither blocks nor log segments found. Creating new log.
I20260811 19:11:15.592532 7519 log.cc:824] T 90276db4edaf4be490cf3382741fe49b P b60c32f9098646edaf34b986f782a645: Log is configured to *not* fsync() on all Append() calls
I20260811 19:11:15.593933 7520 log.cc:824] T 90276db4edaf4be490cf3382741fe49b P 5fa054465f054cdcaef05668dd6c801c: Log is configured to *not* fsync() on all Append() calls
I20260811 19:11:15.595645 7522 tablet_bootstrap.cc:654] T 90276db4edaf4be490cf3382741fe49b P 5943e4a12ecb475db4d131b5047b8f35: Neither blocks nor log segments found. Creating new log.
I20260811 19:11:15.595908 7519 tablet_bootstrap.cc:492] T 90276db4edaf4be490cf3382741fe49b P b60c32f9098646edaf34b986f782a645: No bootstrap required, opened a new log
I20260811 19:11:15.596248 7519 ts_tablet_manager.cc:1397] T 90276db4edaf4be490cf3382741fe49b P b60c32f9098646edaf34b986f782a645: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.007s
I20260811 19:11:15.597045 7522 log.cc:824] T 90276db4edaf4be490cf3382741fe49b P 5943e4a12ecb475db4d131b5047b8f35: Log is configured to *not* fsync() on all Append() calls
I20260811 19:11:15.597297 7520 tablet_bootstrap.cc:492] T 90276db4edaf4be490cf3382741fe49b P 5fa054465f054cdcaef05668dd6c801c: No bootstrap required, opened a new log
I20260811 19:11:15.597611 7520 ts_tablet_manager.cc:1397] T 90276db4edaf4be490cf3382741fe49b P 5fa054465f054cdcaef05668dd6c801c: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 19:11:15.601313 7522 tablet_bootstrap.cc:492] T 90276db4edaf4be490cf3382741fe49b P 5943e4a12ecb475db4d131b5047b8f35: No bootstrap required, opened a new log
I20260811 19:11:15.601787 7522 ts_tablet_manager.cc:1397] T 90276db4edaf4be490cf3382741fe49b P 5943e4a12ecb475db4d131b5047b8f35: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.004s
I20260811 19:11:15.605938 7519 raft_consensus.cc:348] T 90276db4edaf4be490cf3382741fe49b P b60c32f9098646edaf34b986f782a645 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 } } peers { permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 } } peers { permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 } }
I20260811 19:11:15.606350 7519 raft_consensus.cc:374] T 90276db4edaf4be490cf3382741fe49b P b60c32f9098646edaf34b986f782a645 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:11:15.606441 7519 raft_consensus.cc:729] T 90276db4edaf4be490cf3382741fe49b P b60c32f9098646edaf34b986f782a645 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60c32f9098646edaf34b986f782a645, State: Initialized, Role: FOLLOWER
I20260811 19:11:15.607071 7519 consensus_queue.cc:260] T 90276db4edaf4be490cf3382741fe49b P b60c32f9098646edaf34b986f782a645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 } } peers { permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 } } peers { permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 } }
I20260811 19:11:15.608186 7486 heartbeater.cc:503] Master 127.3.61.190:33901 was elected leader, sending a full tablet report...
I20260811 19:11:15.608363 7520 raft_consensus.cc:348] T 90276db4edaf4be490cf3382741fe49b P 5fa054465f054cdcaef05668dd6c801c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 } } peers { permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 } } peers { permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 } }
I20260811 19:11:15.608847 7520 raft_consensus.cc:374] T 90276db4edaf4be490cf3382741fe49b P 5fa054465f054cdcaef05668dd6c801c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:11:15.608959 7520 raft_consensus.cc:729] T 90276db4edaf4be490cf3382741fe49b P 5fa054465f054cdcaef05668dd6c801c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fa054465f054cdcaef05668dd6c801c, State: Initialized, Role: FOLLOWER
I20260811 19:11:15.609622 7520 consensus_queue.cc:260] T 90276db4edaf4be490cf3382741fe49b P 5fa054465f054cdcaef05668dd6c801c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 } } peers { permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 } } peers { permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 } }
I20260811 19:11:15.610585 7519 ts_tablet_manager.cc:1428] T 90276db4edaf4be490cf3382741fe49b P b60c32f9098646edaf34b986f782a645: Time spent starting tablet: real 0.014s user 0.010s sys 0.003s
I20260811 19:11:15.611694 7192 heartbeater.cc:503] Master 127.3.61.190:33901 was elected leader, sending a full tablet report...
I20260811 19:11:15.612607 7522 raft_consensus.cc:348] T 90276db4edaf4be490cf3382741fe49b P 5943e4a12ecb475db4d131b5047b8f35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 } } peers { permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 } } peers { permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 } }
I20260811 19:11:15.613085 7522 raft_consensus.cc:374] T 90276db4edaf4be490cf3382741fe49b P 5943e4a12ecb475db4d131b5047b8f35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:11:15.613179 7522 raft_consensus.cc:729] T 90276db4edaf4be490cf3382741fe49b P 5943e4a12ecb475db4d131b5047b8f35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5943e4a12ecb475db4d131b5047b8f35, State: Initialized, Role: FOLLOWER
I20260811 19:11:15.613844 7520 ts_tablet_manager.cc:1428] T 90276db4edaf4be490cf3382741fe49b P 5fa054465f054cdcaef05668dd6c801c: Time spent starting tablet: real 0.016s user 0.013s sys 0.001s
I20260811 19:11:15.614408 7522 consensus_queue.cc:260] T 90276db4edaf4be490cf3382741fe49b P 5943e4a12ecb475db4d131b5047b8f35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 } } peers { permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 } } peers { permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 } }
I20260811 19:11:15.622516 7522 ts_tablet_manager.cc:1428] T 90276db4edaf4be490cf3382741fe49b P 5943e4a12ecb475db4d131b5047b8f35: Time spent starting tablet: real 0.021s user 0.014s sys 0.005s
W20260811 19:11:15.686573 7487 tablet.cc:2369] T 90276db4edaf4be490cf3382741fe49b P b60c32f9098646edaf34b986f782a645: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:11:15.708530 7538 raft_consensus.cc:482] T 90276db4edaf4be490cf3382741fe49b P 5943e4a12ecb475db4d131b5047b8f35 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:11:15.708886 7538 raft_consensus.cc:504] T 90276db4edaf4be490cf3382741fe49b P 5943e4a12ecb475db4d131b5047b8f35 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 } } peers { permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 } } peers { permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 } }
I20260811 19:11:15.710758 7538 leader_election.cc:288] T 90276db4edaf4be490cf3382741fe49b P 5943e4a12ecb475db4d131b5047b8f35 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60c32f9098646edaf34b986f782a645 (127.3.61.131:39303), 5fa054465f054cdcaef05668dd6c801c (127.3.61.130:38405)
I20260811 19:11:15.719920 7510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:15.711350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:46891 (local address 127.3.61.131:39303)
0811 19:11:15.711636 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:15.711642 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:15.712366 (+ 724us) server_negotiation.cc:407] Connection header received
0811 19:11:15.712553 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:15.712559 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:15.712639 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:15.712713 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:11:15.713334 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:15.714148 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:15.717748 (+ 3600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:15.718658 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:15.718778 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:15.718841 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:15.719508 (+ 667us) server_negotiation.cc:299] Negotiation successful
0811 19:11:15.719654 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260811 19:11:15.719939 7515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:15.711318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:34651 (local address 127.3.61.130:38405)
0811 19:11:15.711650 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:15.711657 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:15.712496 (+ 839us) server_negotiation.cc:407] Connection header received
0811 19:11:15.712643 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:15.712647 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:15.712714 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:15.712802 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:11:15.713425 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:15.714156 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:15.717759 (+ 3603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:15.718740 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:15.718935 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:15.719001 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:15.719508 (+ 507us) server_negotiation.cc:299] Negotiation successful
0811 19:11:15.719669 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156}
I20260811 19:11:15.720141 7543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:15.711258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.131:39303 (local address 127.3.61.129:46891)
0811 19:11:15.712253 (+ 995us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:15.712269 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:15.712411 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:15.712752 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:15.712756 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:15.712781 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:11:15.713183 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:15.713196 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:15.714300 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:15.714304 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:15.717496 (+ 3192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:15.717515 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:15.719039 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:15.719044 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:15.719271 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:15.719330 (+ 59us) client_negotiation.cc:769] Sending connection context
0811 19:11:15.719426 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 19:11:15.719655 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":848,"mutex_wait_us":251,"thread_start_us":677,"threads_started":1}
I20260811 19:11:15.720530 7542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:15.711346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.130:38405 (local address 127.3.61.129:34651)
0811 19:11:15.712366 (+ 1020us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:15.712386 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:15.712523 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:15.712849 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:15.712854 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:15.712887 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:11:15.713266 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:15.713278 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:15.714299 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:15.714304 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:15.717493 (+ 3189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:15.717515 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:15.719016 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:15.719027 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:15.719275 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:15.719323 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 19:11:15.719426 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 19:11:15.719655 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":897,"mutex_wait_us":201,"thread_start_us":434,"threads_started":1}
I20260811 19:11:15.721315 7389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90276db4edaf4be490cf3382741fe49b" candidate_uuid: "5943e4a12ecb475db4d131b5047b8f35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60c32f9098646edaf34b986f782a645" is_pre_election: true
I20260811 19:11:15.721551 7114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90276db4edaf4be490cf3382741fe49b" candidate_uuid: "5943e4a12ecb475db4d131b5047b8f35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fa054465f054cdcaef05668dd6c801c" is_pre_election: true
I20260811 19:11:15.722033 7389 raft_consensus.cc:2454] T 90276db4edaf4be490cf3382741fe49b P b60c32f9098646edaf34b986f782a645 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5943e4a12ecb475db4d131b5047b8f35 in term 0.
I20260811 19:11:15.722054 7114 raft_consensus.cc:2454] T 90276db4edaf4be490cf3382741fe49b P 5fa054465f054cdcaef05668dd6c801c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5943e4a12ecb475db4d131b5047b8f35 in term 0.
I20260811 19:11:15.722884 6689 leader_election.cc:302] T 90276db4edaf4be490cf3382741fe49b P 5943e4a12ecb475db4d131b5047b8f35 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5943e4a12ecb475db4d131b5047b8f35, 5fa054465f054cdcaef05668dd6c801c; no voters:
I20260811 19:11:15.723292 7538 raft_consensus.cc:2781] T 90276db4edaf4be490cf3382741fe49b P 5943e4a12ecb475db4d131b5047b8f35 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:11:15.723461 7538 raft_consensus.cc:482] T 90276db4edaf4be490cf3382741fe49b P 5943e4a12ecb475db4d131b5047b8f35 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:11:15.723563 7538 raft_consensus.cc:3037] T 90276db4edaf4be490cf3382741fe49b P 5943e4a12ecb475db4d131b5047b8f35 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:11:15.726046 7538 raft_consensus.cc:504] T 90276db4edaf4be490cf3382741fe49b P 5943e4a12ecb475db4d131b5047b8f35 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 } } peers { permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 } } peers { permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 } }
I20260811 19:11:15.726977 7538 leader_election.cc:288] T 90276db4edaf4be490cf3382741fe49b P 5943e4a12ecb475db4d131b5047b8f35 [CANDIDATE]: Term 1 election: Requested vote from peers b60c32f9098646edaf34b986f782a645 (127.3.61.131:39303), 5fa054465f054cdcaef05668dd6c801c (127.3.61.130:38405)
I20260811 19:11:15.727638 7114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90276db4edaf4be490cf3382741fe49b" candidate_uuid: "5943e4a12ecb475db4d131b5047b8f35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fa054465f054cdcaef05668dd6c801c"
I20260811 19:11:15.727985 7389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90276db4edaf4be490cf3382741fe49b" candidate_uuid: "5943e4a12ecb475db4d131b5047b8f35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60c32f9098646edaf34b986f782a645"
I20260811 19:11:15.728111 7114 raft_consensus.cc:3037] T 90276db4edaf4be490cf3382741fe49b P 5fa054465f054cdcaef05668dd6c801c [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:11:15.728400 7389 raft_consensus.cc:3037] T 90276db4edaf4be490cf3382741fe49b P b60c32f9098646edaf34b986f782a645 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:11:15.732399 7389 raft_consensus.cc:2454] T 90276db4edaf4be490cf3382741fe49b P b60c32f9098646edaf34b986f782a645 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5943e4a12ecb475db4d131b5047b8f35 in term 1.
I20260811 19:11:15.732401 7114 raft_consensus.cc:2454] T 90276db4edaf4be490cf3382741fe49b P 5fa054465f054cdcaef05668dd6c801c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5943e4a12ecb475db4d131b5047b8f35 in term 1.
I20260811 19:11:15.733045 6689 leader_election.cc:302] T 90276db4edaf4be490cf3382741fe49b P 5943e4a12ecb475db4d131b5047b8f35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5943e4a12ecb475db4d131b5047b8f35, 5fa054465f054cdcaef05668dd6c801c; no voters:
I20260811 19:11:15.733402 7538 raft_consensus.cc:2781] T 90276db4edaf4be490cf3382741fe49b P 5943e4a12ecb475db4d131b5047b8f35 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:11:15.733979 7538 raft_consensus.cc:686] T 90276db4edaf4be490cf3382741fe49b P 5943e4a12ecb475db4d131b5047b8f35 [term 1 LEADER]: Becoming Leader. State: Replica: 5943e4a12ecb475db4d131b5047b8f35, State: Running, Role: LEADER
I20260811 19:11:15.734658 7538 consensus_queue.cc:237] T 90276db4edaf4be490cf3382741fe49b P 5943e4a12ecb475db4d131b5047b8f35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 } } peers { permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 } } peers { permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 } }
I20260811 19:11:15.739506 6535 catalog_manager.cc:5495] T 90276db4edaf4be490cf3382741fe49b P 5943e4a12ecb475db4d131b5047b8f35 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5943e4a12ecb475db4d131b5047b8f35 (127.3.61.129). New cstate: current_term: 1 leader_uuid: "5943e4a12ecb475db4d131b5047b8f35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 } health_report { overall_health: UNKNOWN } } }
W20260811 19:11:15.779440 6926 tablet.cc:2369] T 90276db4edaf4be490cf3382741fe49b P 5943e4a12ecb475db4d131b5047b8f35: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 19:11:15 dist-test-slave-t7g5 krb5kdc[6453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475473, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.190@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:11:15.791492 7249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:15.768853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33546 (local address 127.3.61.190:33901)
0811 19:11:15.769136 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:15.769144 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:15.769643 (+ 499us) server_negotiation.cc:407] Connection header received
0811 19:11:15.770051 (+ 408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:15.770056 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:15.770133 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:15.770246 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:15.772029 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:15.772874 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:15.774636 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:15.775298 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:15.775424 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:15.784707 (+ 9283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:15.784714 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:15.784718 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:15.784757 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:15.788147 (+ 3390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:15.789084 (+ 937us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:15.789090 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:15.789093 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:15.789341 (+ 248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:15.789772 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:15.789777 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:15.789780 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:15.790092 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:15.790227 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:15.790677 (+ 450us) server_negotiation.cc:299] Negotiation successful
0811 19:11:15.791124 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":225,"server-negotiator.queue_time_us":116}
I20260811 19:11:15.791500 7558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:15.768745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33901 (local address 127.0.0.1:33546)
0811 19:11:15.769455 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:15.769473 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:15.769839 (+ 366us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:15.770325 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:15.770329 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:15.771198 (+ 869us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:11:15.771819 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:15.771836 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:15.773056 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:15.773062 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:15.774493 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:15.774512 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:15.775504 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:15.775510 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:15.775664 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:15.781263 (+ 5599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:11:15.781294 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:11:15.784425 (+ 3131us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:11:15.788355 (+ 3930us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:15.788382 (+ 27us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:15.788390 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:15.788963 (+ 573us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:15.789470 (+ 507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:15.789475 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:15.789479 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:15.789639 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:15.790221 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:11:15.790230 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:11:15.790421 (+ 191us) client_negotiation.cc:769] Sending connection context
0811 19:11:15.790564 (+ 143us) client_negotiation.cc:240] Negotiation successful
0811 19:11:15.790961 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":417,"threads_started":1}
I20260811 19:11:15.797485 6535 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:33546:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 19:11:15.798676 6535 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:11:15.807811 6791 tablet_service.cc:1467] Processing CreateTablet for tablet 71f17636b3e64cb1acf7303a8f8886d7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0fe3eb05c53c4fe2930f295e62b0cb22]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 19:11:15.807900 7068 tablet_service.cc:1467] Processing CreateTablet for tablet 71f17636b3e64cb1acf7303a8f8886d7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0fe3eb05c53c4fe2930f295e62b0cb22]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 19:11:15.807957 7338 tablet_service.cc:1467] Processing CreateTablet for tablet 71f17636b3e64cb1acf7303a8f8886d7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0fe3eb05c53c4fe2930f295e62b0cb22]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 19:11:15.808789 6791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71f17636b3e64cb1acf7303a8f8886d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:11:15.808983 7338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71f17636b3e64cb1acf7303a8f8886d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:11:15.808995 7068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71f17636b3e64cb1acf7303a8f8886d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:11:15.814503 7522 tablet_bootstrap.cc:492] T 71f17636b3e64cb1acf7303a8f8886d7 P 5943e4a12ecb475db4d131b5047b8f35: Bootstrap starting.
I20260811 19:11:15.816318 7522 tablet_bootstrap.cc:654] T 71f17636b3e64cb1acf7303a8f8886d7 P 5943e4a12ecb475db4d131b5047b8f35: Neither blocks nor log segments found. Creating new log.
I20260811 19:11:15.816895 7519 tablet_bootstrap.cc:492] T 71f17636b3e64cb1acf7303a8f8886d7 P b60c32f9098646edaf34b986f782a645: Bootstrap starting.
I20260811 19:11:15.816910 7520 tablet_bootstrap.cc:492] T 71f17636b3e64cb1acf7303a8f8886d7 P 5fa054465f054cdcaef05668dd6c801c: Bootstrap starting.
I20260811 19:11:15.818876 7520 tablet_bootstrap.cc:654] T 71f17636b3e64cb1acf7303a8f8886d7 P 5fa054465f054cdcaef05668dd6c801c: Neither blocks nor log segments found. Creating new log.
I20260811 19:11:15.818871 7519 tablet_bootstrap.cc:654] T 71f17636b3e64cb1acf7303a8f8886d7 P b60c32f9098646edaf34b986f782a645: Neither blocks nor log segments found. Creating new log.
I20260811 19:11:15.820775 7522 tablet_bootstrap.cc:492] T 71f17636b3e64cb1acf7303a8f8886d7 P 5943e4a12ecb475db4d131b5047b8f35: No bootstrap required, opened a new log
I20260811 19:11:15.820955 7522 ts_tablet_manager.cc:1397] T 71f17636b3e64cb1acf7303a8f8886d7 P 5943e4a12ecb475db4d131b5047b8f35: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.002s
I20260811 19:11:15.821321 7520 tablet_bootstrap.cc:492] T 71f17636b3e64cb1acf7303a8f8886d7 P 5fa054465f054cdcaef05668dd6c801c: No bootstrap required, opened a new log
I20260811 19:11:15.821491 7520 ts_tablet_manager.cc:1397] T 71f17636b3e64cb1acf7303a8f8886d7 P 5fa054465f054cdcaef05668dd6c801c: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.002s
I20260811 19:11:15.821861 7522 raft_consensus.cc:348] T 71f17636b3e64cb1acf7303a8f8886d7 P 5943e4a12ecb475db4d131b5047b8f35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 } } peers { permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 } } peers { permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 } }
I20260811 19:11:15.822069 7522 raft_consensus.cc:374] T 71f17636b3e64cb1acf7303a8f8886d7 P 5943e4a12ecb475db4d131b5047b8f35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:11:15.822104 7519 tablet_bootstrap.cc:492] T 71f17636b3e64cb1acf7303a8f8886d7 P b60c32f9098646edaf34b986f782a645: No bootstrap required, opened a new log
I20260811 19:11:15.822163 7522 raft_consensus.cc:729] T 71f17636b3e64cb1acf7303a8f8886d7 P 5943e4a12ecb475db4d131b5047b8f35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5943e4a12ecb475db4d131b5047b8f35, State: Initialized, Role: FOLLOWER
I20260811 19:11:15.822268 7519 ts_tablet_manager.cc:1397] T 71f17636b3e64cb1acf7303a8f8886d7 P b60c32f9098646edaf34b986f782a645: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.003s
I20260811 19:11:15.822364 7520 raft_consensus.cc:348] T 71f17636b3e64cb1acf7303a8f8886d7 P 5fa054465f054cdcaef05668dd6c801c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 } } peers { permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 } } peers { permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 } }
I20260811 19:11:15.822455 7522 consensus_queue.cc:260] T 71f17636b3e64cb1acf7303a8f8886d7 P 5943e4a12ecb475db4d131b5047b8f35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 } } peers { permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 } } peers { permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 } }
I20260811 19:11:15.822614 7520 raft_consensus.cc:374] T 71f17636b3e64cb1acf7303a8f8886d7 P 5fa054465f054cdcaef05668dd6c801c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:11:15.822700 7520 raft_consensus.cc:729] T 71f17636b3e64cb1acf7303a8f8886d7 P 5fa054465f054cdcaef05668dd6c801c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fa054465f054cdcaef05668dd6c801c, State: Initialized, Role: FOLLOWER
I20260811 19:11:15.822925 7538 tablet_replica.cc:439] T 71f17636b3e64cb1acf7303a8f8886d7 P 5943e4a12ecb475db4d131b5047b8f35: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 } } peers { permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 } } peers { permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 } } }
I20260811 19:11:15.822994 7520 consensus_queue.cc:260] T 71f17636b3e64cb1acf7303a8f8886d7 P 5fa054465f054cdcaef05668dd6c801c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 } } peers { permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 } } peers { permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 } }
I20260811 19:11:15.823225 7538 tablet_replica.cc:442] T 71f17636b3e64cb1acf7303a8f8886d7 P 5943e4a12ecb475db4d131b5047b8f35: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 19:11:15.823096 7519 raft_consensus.cc:348] T 71f17636b3e64cb1acf7303a8f8886d7 P b60c32f9098646edaf34b986f782a645 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 } } peers { permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 } } peers { permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 } }
I20260811 19:11:15.823342 7519 raft_consensus.cc:374] T 71f17636b3e64cb1acf7303a8f8886d7 P b60c32f9098646edaf34b986f782a645 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:11:15.823448 7519 raft_consensus.cc:729] T 71f17636b3e64cb1acf7303a8f8886d7 P b60c32f9098646edaf34b986f782a645 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60c32f9098646edaf34b986f782a645, State: Initialized, Role: FOLLOWER
I20260811 19:11:15.823458 7532 tablet_replica.cc:439] T 71f17636b3e64cb1acf7303a8f8886d7 P 5fa054465f054cdcaef05668dd6c801c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 } } peers { permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 } } peers { permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 } } }
I20260811 19:11:15.823793 7532 tablet_replica.cc:442] T 71f17636b3e64cb1acf7303a8f8886d7 P 5fa054465f054cdcaef05668dd6c801c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 19:11:15.823770 7519 consensus_queue.cc:260] T 71f17636b3e64cb1acf7303a8f8886d7 P b60c32f9098646edaf34b986f782a645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 } } peers { permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 } } peers { permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 } }
I20260811 19:11:15.824204 7531 tablet_replica.cc:439] T 71f17636b3e64cb1acf7303a8f8886d7 P b60c32f9098646edaf34b986f782a645: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 } } peers { permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 } } peers { permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 } } }
I20260811 19:11:15.824544 7531 tablet_replica.cc:442] T 71f17636b3e64cb1acf7303a8f8886d7 P b60c32f9098646edaf34b986f782a645: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 19:11:15.824779 7520 ts_tablet_manager.cc:1428] T 71f17636b3e64cb1acf7303a8f8886d7 P 5fa054465f054cdcaef05668dd6c801c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 19:11:15.825295 7522 ts_tablet_manager.cc:1428] T 71f17636b3e64cb1acf7303a8f8886d7 P 5943e4a12ecb475db4d131b5047b8f35: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260811 19:11:15.825914 7519 ts_tablet_manager.cc:1428] T 71f17636b3e64cb1acf7303a8f8886d7 P b60c32f9098646edaf34b986f782a645: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
W20260811 19:11:15.851958 7194 tablet.cc:2369] T 71f17636b3e64cb1acf7303a8f8886d7 P 5fa054465f054cdcaef05668dd6c801c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:11:15.908646 7538 raft_consensus.cc:482] T 71f17636b3e64cb1acf7303a8f8886d7 P 5943e4a12ecb475db4d131b5047b8f35 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:11:15.909088 7538 raft_consensus.cc:504] T 71f17636b3e64cb1acf7303a8f8886d7 P 5943e4a12ecb475db4d131b5047b8f35 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 } } peers { permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 } } peers { permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 } }
I20260811 19:11:15.910109 7538 leader_election.cc:288] T 71f17636b3e64cb1acf7303a8f8886d7 P 5943e4a12ecb475db4d131b5047b8f35 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5fa054465f054cdcaef05668dd6c801c (127.3.61.130:38405), b60c32f9098646edaf34b986f782a645 (127.3.61.131:39303)
I20260811 19:11:15.910779 7114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71f17636b3e64cb1acf7303a8f8886d7" candidate_uuid: "5943e4a12ecb475db4d131b5047b8f35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fa054465f054cdcaef05668dd6c801c" is_pre_election: true
I20260811 19:11:15.910877 7389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71f17636b3e64cb1acf7303a8f8886d7" candidate_uuid: "5943e4a12ecb475db4d131b5047b8f35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60c32f9098646edaf34b986f782a645" is_pre_election: true
I20260811 19:11:15.911052 7114 raft_consensus.cc:2454] T 71f17636b3e64cb1acf7303a8f8886d7 P 5fa054465f054cdcaef05668dd6c801c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5943e4a12ecb475db4d131b5047b8f35 in term 0.
I20260811 19:11:15.911216 7389 raft_consensus.cc:2454] T 71f17636b3e64cb1acf7303a8f8886d7 P b60c32f9098646edaf34b986f782a645 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5943e4a12ecb475db4d131b5047b8f35 in term 0.
I20260811 19:11:15.911549 6689 leader_election.cc:302] T 71f17636b3e64cb1acf7303a8f8886d7 P 5943e4a12ecb475db4d131b5047b8f35 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5943e4a12ecb475db4d131b5047b8f35, 5fa054465f054cdcaef05668dd6c801c; no voters:
I20260811 19:11:15.911897 7538 raft_consensus.cc:2781] T 71f17636b3e64cb1acf7303a8f8886d7 P 5943e4a12ecb475db4d131b5047b8f35 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:11:15.912009 7538 raft_consensus.cc:482] T 71f17636b3e64cb1acf7303a8f8886d7 P 5943e4a12ecb475db4d131b5047b8f35 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:11:15.912093 7538 raft_consensus.cc:3037] T 71f17636b3e64cb1acf7303a8f8886d7 P 5943e4a12ecb475db4d131b5047b8f35 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:11:15.914242 7538 raft_consensus.cc:504] T 71f17636b3e64cb1acf7303a8f8886d7 P 5943e4a12ecb475db4d131b5047b8f35 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 } } peers { permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 } } peers { permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 } }
I20260811 19:11:15.914844 7538 leader_election.cc:288] T 71f17636b3e64cb1acf7303a8f8886d7 P 5943e4a12ecb475db4d131b5047b8f35 [CANDIDATE]: Term 1 election: Requested vote from peers 5fa054465f054cdcaef05668dd6c801c (127.3.61.130:38405), b60c32f9098646edaf34b986f782a645 (127.3.61.131:39303)
I20260811 19:11:15.915409 7114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71f17636b3e64cb1acf7303a8f8886d7" candidate_uuid: "5943e4a12ecb475db4d131b5047b8f35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fa054465f054cdcaef05668dd6c801c"
I20260811 19:11:15.915627 7389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71f17636b3e64cb1acf7303a8f8886d7" candidate_uuid: "5943e4a12ecb475db4d131b5047b8f35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60c32f9098646edaf34b986f782a645"
I20260811 19:11:15.915690 7114 raft_consensus.cc:3037] T 71f17636b3e64cb1acf7303a8f8886d7 P 5fa054465f054cdcaef05668dd6c801c [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:11:15.915863 7389 raft_consensus.cc:3037] T 71f17636b3e64cb1acf7303a8f8886d7 P b60c32f9098646edaf34b986f782a645 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:11:15.918208 7389 raft_consensus.cc:2454] T 71f17636b3e64cb1acf7303a8f8886d7 P b60c32f9098646edaf34b986f782a645 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5943e4a12ecb475db4d131b5047b8f35 in term 1.
I20260811 19:11:15.918710 6691 leader_election.cc:302] T 71f17636b3e64cb1acf7303a8f8886d7 P 5943e4a12ecb475db4d131b5047b8f35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5943e4a12ecb475db4d131b5047b8f35, b60c32f9098646edaf34b986f782a645; no voters:
I20260811 19:11:15.919026 7538 raft_consensus.cc:2781] T 71f17636b3e64cb1acf7303a8f8886d7 P 5943e4a12ecb475db4d131b5047b8f35 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:11:15.919134 7114 raft_consensus.cc:2454] T 71f17636b3e64cb1acf7303a8f8886d7 P 5fa054465f054cdcaef05668dd6c801c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5943e4a12ecb475db4d131b5047b8f35 in term 1.
I20260811 19:11:15.919329 7538 raft_consensus.cc:686] T 71f17636b3e64cb1acf7303a8f8886d7 P 5943e4a12ecb475db4d131b5047b8f35 [term 1 LEADER]: Becoming Leader. State: Replica: 5943e4a12ecb475db4d131b5047b8f35, State: Running, Role: LEADER
I20260811 19:11:15.919652 7538 consensus_queue.cc:237] T 71f17636b3e64cb1acf7303a8f8886d7 P 5943e4a12ecb475db4d131b5047b8f35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 } } peers { permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 } } peers { permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 } }
I20260811 19:11:15.921373 7546 tablet_replica.cc:439] T 71f17636b3e64cb1acf7303a8f8886d7 P 5943e4a12ecb475db4d131b5047b8f35: TxnStatusTablet state changed. Reason: New leader 5943e4a12ecb475db4d131b5047b8f35. Latest consensus state: current_term: 1 leader_uuid: "5943e4a12ecb475db4d131b5047b8f35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 } } peers { permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 } } peers { permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 } } }
I20260811 19:11:15.921794 7546 tablet_replica.cc:442] T 71f17636b3e64cb1acf7303a8f8886d7 P 5943e4a12ecb475db4d131b5047b8f35: This TxnStatusTablet replica's current role is: LEADER
I20260811 19:11:15.922690 7569 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 19:11:15.923041 7569 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 19:11:15.923048 6530 catalog_manager.cc:5495] T 71f17636b3e64cb1acf7303a8f8886d7 P 5943e4a12ecb475db4d131b5047b8f35 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5943e4a12ecb475db4d131b5047b8f35 (127.3.61.129). New cstate: current_term: 1 leader_uuid: "5943e4a12ecb475db4d131b5047b8f35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 } health_report { overall_health: UNKNOWN } } }
I20260811 19:11:15.960952 7558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:15.953884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:46395 (local address 127.0.0.1:39450)
0811 19:11:15.954294 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:15.954316 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:15.954452 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:15.955004 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:15.955008 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:15.955018 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 19:11:15.955188 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:15.955195 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:15.956716 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:15.956720 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:15.957977 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:15.957992 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:15.959036 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:15.959041 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:15.959220 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:15.959290 (+ 70us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 19:11:15.960436 (+ 1146us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 19:11:15.960458 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 19:11:15.960541 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 19:11:15.960682 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119}
I20260811 19:11:15.960999 7511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:15.954169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39450 (local address 127.3.61.129:46395)
0811 19:11:15.954413 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:15.954420 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:15.954445 (+ 25us) server_negotiation.cc:407] Connection header received
0811 19:11:15.954653 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:15.954659 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:15.954794 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:15.954958 (+ 164us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:11:15.955331 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:15.956584 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:15.958125 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:15.958836 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:15.959007 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:15.959524 (+ 517us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:11:15.960285 (+ 761us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:11:15.960446 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:15.960588 (+ 142us) server_negotiation.cc:299] Negotiation successful
0811 19:11:15.960682 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 19:11:15.969231 7389 raft_consensus.cc:1261] T 71f17636b3e64cb1acf7303a8f8886d7 P b60c32f9098646edaf34b986f782a645 [term 1 FOLLOWER]: Refusing update from remote peer 5943e4a12ecb475db4d131b5047b8f35: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:11:15.969385 7114 raft_consensus.cc:1261] T 71f17636b3e64cb1acf7303a8f8886d7 P 5fa054465f054cdcaef05668dd6c801c [term 1 FOLLOWER]: Refusing update from remote peer 5943e4a12ecb475db4d131b5047b8f35: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:11:15.970412 7538 consensus_queue.cc:1035] T 71f17636b3e64cb1acf7303a8f8886d7 P 5943e4a12ecb475db4d131b5047b8f35 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:11:15.970834 7546 consensus_queue.cc:1035] T 71f17636b3e64cb1acf7303a8f8886d7 P 5943e4a12ecb475db4d131b5047b8f35 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:11:15.979125 7531 tablet_replica.cc:439] T 71f17636b3e64cb1acf7303a8f8886d7 P b60c32f9098646edaf34b986f782a645: TxnStatusTablet state changed. Reason: New leader 5943e4a12ecb475db4d131b5047b8f35. Latest consensus state: current_term: 1 leader_uuid: "5943e4a12ecb475db4d131b5047b8f35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 } } peers { permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 } } peers { permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 } } }
I20260811 19:11:15.979478 7531 tablet_replica.cc:442] T 71f17636b3e64cb1acf7303a8f8886d7 P b60c32f9098646edaf34b986f782a645: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 19:11:15.982640 7580 mvcc.cc:205] Tried to move back new op lower bound from 7317403549559767040 to 7317403549369479168. Current Snapshot: MvccSnapshot[applied={T|T < 7317403549559767040}]
I20260811 19:11:15.983451 7538 tablet_replica.cc:439] T 71f17636b3e64cb1acf7303a8f8886d7 P 5943e4a12ecb475db4d131b5047b8f35: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5943e4a12ecb475db4d131b5047b8f35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 } } peers { permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 } } peers { permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 } } }
I20260811 19:11:15.983809 7538 tablet_replica.cc:442] T 71f17636b3e64cb1acf7303a8f8886d7 P 5943e4a12ecb475db4d131b5047b8f35: This TxnStatusTablet replica's current role is: LEADER
I20260811 19:11:15.992640 7532 tablet_replica.cc:439] T 71f17636b3e64cb1acf7303a8f8886d7 P 5fa054465f054cdcaef05668dd6c801c: TxnStatusTablet state changed. Reason: New leader 5943e4a12ecb475db4d131b5047b8f35. Latest consensus state: current_term: 1 leader_uuid: "5943e4a12ecb475db4d131b5047b8f35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 } } peers { permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 } } peers { permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 } } }
I20260811 19:11:15.993006 7532 tablet_replica.cc:442] T 71f17636b3e64cb1acf7303a8f8886d7 P 5fa054465f054cdcaef05668dd6c801c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 19:11:15.998327 7546 tablet_replica.cc:439] T 71f17636b3e64cb1acf7303a8f8886d7 P 5943e4a12ecb475db4d131b5047b8f35: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5943e4a12ecb475db4d131b5047b8f35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 } } peers { permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 } } peers { permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 } } }
I20260811 19:11:15.998670 7546 tablet_replica.cc:442] T 71f17636b3e64cb1acf7303a8f8886d7 P 5943e4a12ecb475db4d131b5047b8f35: This TxnStatusTablet replica's current role is: LEADER
I20260811 19:11:16.000209 7531 tablet_replica.cc:439] T 71f17636b3e64cb1acf7303a8f8886d7 P b60c32f9098646edaf34b986f782a645: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5943e4a12ecb475db4d131b5047b8f35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 } } peers { permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 } } peers { permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 } } }
I20260811 19:11:16.000478 7531 tablet_replica.cc:442] T 71f17636b3e64cb1acf7303a8f8886d7 P b60c32f9098646edaf34b986f782a645: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 19:11:16.005035 7532 tablet_replica.cc:439] T 71f17636b3e64cb1acf7303a8f8886d7 P 5fa054465f054cdcaef05668dd6c801c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5943e4a12ecb475db4d131b5047b8f35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 } } peers { permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 } } peers { permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 } } }
I20260811 19:11:16.005331 7532 tablet_replica.cc:442] T 71f17636b3e64cb1acf7303a8f8886d7 P 5fa054465f054cdcaef05668dd6c801c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 19:11:16.012528 7511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:16.005842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39452 (local address 127.3.61.129:46395)
0811 19:11:16.006010 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:16.006015 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:16.006036 (+ 21us) server_negotiation.cc:407] Connection header received
0811 19:11:16.006242 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:16.006247 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:16.006442 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:16.006543 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:11:16.007273 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:16.008502 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:16.010572 (+ 2070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:16.011020 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:16.011119 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:16.011692 (+ 573us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:11:16.011822 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:11:16.011908 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:16.012104 (+ 196us) server_negotiation.cc:299] Negotiation successful
0811 19:11:16.012182 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 19:11:16.032605 7599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:16.025025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:46395 (local address 127.3.61.129:45293)
0811 19:11:16.025946 (+ 921us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:16.025964 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:16.026101 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:16.026481 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:16.026485 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:16.026495 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 19:11:16.026840 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:16.026848 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:16.028424 (+ 1576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:16.028428 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:16.030009 (+ 1581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:16.030027 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:16.031197 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:16.031205 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:16.031400 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:16.031485 (+ 85us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 19:11:16.032000 (+ 515us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 19:11:16.032013 (+ 13us) client_negotiation.cc:769] Sending connection context
0811 19:11:16.032100 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 19:11:16.032260 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":730,"thread_start_us":549,"threads_started":1}
I20260811 19:11:16.032629 7511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:16.025025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:45293 (local address 127.3.61.129:46395)
0811 19:11:16.025258 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:16.025265 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:16.026056 (+ 791us) server_negotiation.cc:407] Connection header received
0811 19:11:16.026232 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:16.026239 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:16.026340 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:16.026464 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:11:16.027004 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:16.028214 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:16.030181 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:16.030949 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:16.031096 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:16.031706 (+ 610us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:11:16.031866 (+ 160us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:11:16.031976 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:16.032198 (+ 222us) server_negotiation.cc:299] Negotiation successful
0811 19:11:16.032300 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 19:11:16.043296 7389 raft_consensus.cc:1261] T 90276db4edaf4be490cf3382741fe49b P b60c32f9098646edaf34b986f782a645 [term 1 FOLLOWER]: Refusing update from remote peer 5943e4a12ecb475db4d131b5047b8f35: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:11:16.043643 7114 raft_consensus.cc:1261] T 90276db4edaf4be490cf3382741fe49b P 5fa054465f054cdcaef05668dd6c801c [term 1 FOLLOWER]: Refusing update from remote peer 5943e4a12ecb475db4d131b5047b8f35: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:11:16.043957 7600 consensus_queue.cc:1035] T 90276db4edaf4be490cf3382741fe49b P 5943e4a12ecb475db4d131b5047b8f35 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:11:16.044411 7546 consensus_queue.cc:1035] T 90276db4edaf4be490cf3382741fe49b P 5943e4a12ecb475db4d131b5047b8f35 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:11:16.129153 6530 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33540:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:11:16.129511 6530 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33540:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:11:16.133837 6530 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 404597c6971d4297830d78a22a801cac: Sending DeleteTablet for 3 replicas of tablet 90276db4edaf4be490cf3382741fe49b
I20260811 19:11:16.135171 7338 tablet_service.cc:1514] Processing DeleteTablet for tablet 90276db4edaf4be490cf3382741fe49b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:11:16 UTC) from {username='kudu', principal='kudu/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:58612
I20260811 19:11:16.136591 6530 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33540:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 19:11:16.137321 6530 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:11:16.137964 7068 tablet_service.cc:1514] Processing DeleteTablet for tablet 90276db4edaf4be490cf3382741fe49b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:11:16 UTC) from {username='kudu', principal='kudu/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:35282
I20260811 19:11:16.138912 7609 tablet_replica.cc:330] T 90276db4edaf4be490cf3382741fe49b P 5fa054465f054cdcaef05668dd6c801c: stopping tablet replica
I20260811 19:11:16.139379 7609 raft_consensus.cc:2229] T 90276db4edaf4be490cf3382741fe49b P 5fa054465f054cdcaef05668dd6c801c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:11:16.139775 7609 raft_consensus.cc:2258] T 90276db4edaf4be490cf3382741fe49b P 5fa054465f054cdcaef05668dd6c801c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:11:16.142902 6791 tablet_service.cc:1514] Processing DeleteTablet for tablet 90276db4edaf4be490cf3382741fe49b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:11:16 UTC) from {username='kudu', principal='kudu/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:39448
I20260811 19:11:16.150156 7610 tablet_replica.cc:330] T 90276db4edaf4be490cf3382741fe49b P 5943e4a12ecb475db4d131b5047b8f35: stopping tablet replica
I20260811 19:11:16.150695 7610 raft_consensus.cc:2229] T 90276db4edaf4be490cf3382741fe49b P 5943e4a12ecb475db4d131b5047b8f35 [term 1 LEADER]: Raft consensus shutting down.
I20260811 19:11:16.151394 7610 raft_consensus.cc:2258] T 90276db4edaf4be490cf3382741fe49b P 5943e4a12ecb475db4d131b5047b8f35 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:11:16.152644 6791 tablet_service.cc:1467] Processing CreateTablet for tablet 85cf312a02384676a6f6a60dd0d5dc9f (DEFAULT_TABLE table=test-table [id=14f09958ee1f407d9d3d6c2171b70148]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:11:16.152827 7335 tablet_service.cc:1467] Processing CreateTablet for tablet 85cf312a02384676a6f6a60dd0d5dc9f (DEFAULT_TABLE table=test-table [id=14f09958ee1f407d9d3d6c2171b70148]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:11:16.153136 6791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85cf312a02384676a6f6a60dd0d5dc9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:11:16.153335 7335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85cf312a02384676a6f6a60dd0d5dc9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:11:16.156816 7609 ts_tablet_manager.cc:1905] T 90276db4edaf4be490cf3382741fe49b P 5fa054465f054cdcaef05668dd6c801c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:11:16.157543 7068 tablet_service.cc:1467] Processing CreateTablet for tablet 85cf312a02384676a6f6a60dd0d5dc9f (DEFAULT_TABLE table=test-table [id=14f09958ee1f407d9d3d6c2171b70148]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:11:16.158035 7068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85cf312a02384676a6f6a60dd0d5dc9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:11:16.159294 7610 ts_tablet_manager.cc:1905] T 90276db4edaf4be490cf3382741fe49b P 5943e4a12ecb475db4d131b5047b8f35: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:11:16.160216 7522 tablet_bootstrap.cc:492] T 85cf312a02384676a6f6a60dd0d5dc9f P 5943e4a12ecb475db4d131b5047b8f35: Bootstrap starting.
I20260811 19:11:16.161901 7522 tablet_bootstrap.cc:654] T 85cf312a02384676a6f6a60dd0d5dc9f P 5943e4a12ecb475db4d131b5047b8f35: Neither blocks nor log segments found. Creating new log.
I20260811 19:11:16.164080 7522 tablet_bootstrap.cc:492] T 85cf312a02384676a6f6a60dd0d5dc9f P 5943e4a12ecb475db4d131b5047b8f35: No bootstrap required, opened a new log
I20260811 19:11:16.164253 7522 ts_tablet_manager.cc:1397] T 85cf312a02384676a6f6a60dd0d5dc9f P 5943e4a12ecb475db4d131b5047b8f35: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260811 19:11:16.165021 7522 raft_consensus.cc:348] T 85cf312a02384676a6f6a60dd0d5dc9f P 5943e4a12ecb475db4d131b5047b8f35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 } } peers { permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 } } peers { permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 } }
I20260811 19:11:16.165306 7519 tablet_bootstrap.cc:492] T 85cf312a02384676a6f6a60dd0d5dc9f P b60c32f9098646edaf34b986f782a645: Bootstrap starting.
I20260811 19:11:16.165306 7522 raft_consensus.cc:374] T 85cf312a02384676a6f6a60dd0d5dc9f P 5943e4a12ecb475db4d131b5047b8f35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:11:16.165391 7522 raft_consensus.cc:729] T 85cf312a02384676a6f6a60dd0d5dc9f P 5943e4a12ecb475db4d131b5047b8f35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5943e4a12ecb475db4d131b5047b8f35, State: Initialized, Role: FOLLOWER
I20260811 19:11:16.165683 7522 consensus_queue.cc:260] T 85cf312a02384676a6f6a60dd0d5dc9f P 5943e4a12ecb475db4d131b5047b8f35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 } } peers { permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 } } peers { permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 } }
I20260811 19:11:16.166930 7522 ts_tablet_manager.cc:1428] T 85cf312a02384676a6f6a60dd0d5dc9f P 5943e4a12ecb475db4d131b5047b8f35: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260811 19:11:16.167032 7519 tablet_bootstrap.cc:654] T 85cf312a02384676a6f6a60dd0d5dc9f P b60c32f9098646edaf34b986f782a645: Neither blocks nor log segments found. Creating new log.
I20260811 19:11:16.167984 7609 ts_tablet_manager.cc:1918] T 90276db4edaf4be490cf3382741fe49b P 5fa054465f054cdcaef05668dd6c801c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 19:11:16.168184 7609 log.cc:1196] T 90276db4edaf4be490cf3382741fe49b P 5fa054465f054cdcaef05668dd6c801c: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-1/wal/wals/90276db4edaf4be490cf3382741fe49b
I20260811 19:11:16.169170 7609 ts_tablet_manager.cc:1939] T 90276db4edaf4be490cf3382741fe49b P 5fa054465f054cdcaef05668dd6c801c: Deleting consensus metadata
I20260811 19:11:16.172580 7520 tablet_bootstrap.cc:492] T 85cf312a02384676a6f6a60dd0d5dc9f P 5fa054465f054cdcaef05668dd6c801c: Bootstrap starting.
I20260811 19:11:16.174628 7520 tablet_bootstrap.cc:654] T 85cf312a02384676a6f6a60dd0d5dc9f P 5fa054465f054cdcaef05668dd6c801c: Neither blocks nor log segments found. Creating new log.
I20260811 19:11:16.174935 7608 tablet_replica.cc:330] T 90276db4edaf4be490cf3382741fe49b P b60c32f9098646edaf34b986f782a645: stopping tablet replica
I20260811 19:11:16.175616 7608 raft_consensus.cc:2229] T 90276db4edaf4be490cf3382741fe49b P b60c32f9098646edaf34b986f782a645 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:11:16.176128 7608 raft_consensus.cc:2258] T 90276db4edaf4be490cf3382741fe49b P b60c32f9098646edaf34b986f782a645 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:11:16.178622 7608 ts_tablet_manager.cc:1905] T 90276db4edaf4be490cf3382741fe49b P b60c32f9098646edaf34b986f782a645: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:11:16.180563 6503 catalog_manager.cc:4841] TS 5fa054465f054cdcaef05668dd6c801c (127.3.61.130:38405): tablet 90276db4edaf4be490cf3382741fe49b (table test-table [id=b82fea74a5fe4cf3a7a3121c8d363df8]) successfully deleted
I20260811 19:11:16.184412 7610 ts_tablet_manager.cc:1918] T 90276db4edaf4be490cf3382741fe49b P 5943e4a12ecb475db4d131b5047b8f35: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 19:11:16.184610 7610 log.cc:1196] T 90276db4edaf4be490cf3382741fe49b P 5943e4a12ecb475db4d131b5047b8f35: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-0/wal/wals/90276db4edaf4be490cf3382741fe49b
I20260811 19:11:16.185177 7610 ts_tablet_manager.cc:1939] T 90276db4edaf4be490cf3382741fe49b P 5943e4a12ecb475db4d131b5047b8f35: Deleting consensus metadata
I20260811 19:11:16.187281 7520 tablet_bootstrap.cc:492] T 85cf312a02384676a6f6a60dd0d5dc9f P 5fa054465f054cdcaef05668dd6c801c: No bootstrap required, opened a new log
I20260811 19:11:16.187299 6503 catalog_manager.cc:4841] TS 5943e4a12ecb475db4d131b5047b8f35 (127.3.61.129:46395): tablet 90276db4edaf4be490cf3382741fe49b (table test-table [id=b82fea74a5fe4cf3a7a3121c8d363df8]) successfully deleted
I20260811 19:11:16.187482 7520 ts_tablet_manager.cc:1397] T 85cf312a02384676a6f6a60dd0d5dc9f P 5fa054465f054cdcaef05668dd6c801c: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.003s
I20260811 19:11:16.187685 7608 ts_tablet_manager.cc:1918] T 90276db4edaf4be490cf3382741fe49b P b60c32f9098646edaf34b986f782a645: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 19:11:16.187922 7608 log.cc:1196] T 90276db4edaf4be490cf3382741fe49b P b60c32f9098646edaf34b986f782a645: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-2/wal/wals/90276db4edaf4be490cf3382741fe49b
I20260811 19:11:16.188773 7608 ts_tablet_manager.cc:1939] T 90276db4edaf4be490cf3382741fe49b P b60c32f9098646edaf34b986f782a645: Deleting consensus metadata
I20260811 19:11:16.189153 7519 tablet_bootstrap.cc:492] T 85cf312a02384676a6f6a60dd0d5dc9f P b60c32f9098646edaf34b986f782a645: No bootstrap required, opened a new log
I20260811 19:11:16.189330 7519 ts_tablet_manager.cc:1397] T 85cf312a02384676a6f6a60dd0d5dc9f P b60c32f9098646edaf34b986f782a645: Time spent bootstrapping tablet: real 0.024s user 0.012s sys 0.004s
I20260811 19:11:16.190186 7519 raft_consensus.cc:348] T 85cf312a02384676a6f6a60dd0d5dc9f P b60c32f9098646edaf34b986f782a645 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 } } peers { permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 } } peers { permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 } }
I20260811 19:11:16.190446 7519 raft_consensus.cc:374] T 85cf312a02384676a6f6a60dd0d5dc9f P b60c32f9098646edaf34b986f782a645 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:11:16.190526 7519 raft_consensus.cc:729] T 85cf312a02384676a6f6a60dd0d5dc9f P b60c32f9098646edaf34b986f782a645 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60c32f9098646edaf34b986f782a645, State: Initialized, Role: FOLLOWER
I20260811 19:11:16.190796 7519 consensus_queue.cc:260] T 85cf312a02384676a6f6a60dd0d5dc9f P b60c32f9098646edaf34b986f782a645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 } } peers { permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 } } peers { permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 } }
I20260811 19:11:16.191344 7519 ts_tablet_manager.cc:1428] T 85cf312a02384676a6f6a60dd0d5dc9f P b60c32f9098646edaf34b986f782a645: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260811 19:11:16.191881 6504 catalog_manager.cc:4841] TS b60c32f9098646edaf34b986f782a645 (127.3.61.131:39303): tablet 90276db4edaf4be490cf3382741fe49b (table test-table [id=b82fea74a5fe4cf3a7a3121c8d363df8]) successfully deleted
I20260811 19:11:16.196636 7520 raft_consensus.cc:348] T 85cf312a02384676a6f6a60dd0d5dc9f P 5fa054465f054cdcaef05668dd6c801c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 } } peers { permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 } } peers { permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 } }
I20260811 19:11:16.196942 7520 raft_consensus.cc:374] T 85cf312a02384676a6f6a60dd0d5dc9f P 5fa054465f054cdcaef05668dd6c801c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:11:16.197043 7520 raft_consensus.cc:729] T 85cf312a02384676a6f6a60dd0d5dc9f P 5fa054465f054cdcaef05668dd6c801c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fa054465f054cdcaef05668dd6c801c, State: Initialized, Role: FOLLOWER
I20260811 19:11:16.197361 7520 consensus_queue.cc:260] T 85cf312a02384676a6f6a60dd0d5dc9f P 5fa054465f054cdcaef05668dd6c801c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 } } peers { permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 } } peers { permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 } }
I20260811 19:11:16.198141 7520 ts_tablet_manager.cc:1428] T 85cf312a02384676a6f6a60dd0d5dc9f P 5fa054465f054cdcaef05668dd6c801c: Time spent starting tablet: real 0.011s user 0.002s sys 0.000s
I20260811 19:11:16.304178 7532 raft_consensus.cc:482] T 85cf312a02384676a6f6a60dd0d5dc9f P 5fa054465f054cdcaef05668dd6c801c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:11:16.304529 7532 raft_consensus.cc:504] T 85cf312a02384676a6f6a60dd0d5dc9f P 5fa054465f054cdcaef05668dd6c801c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 } } peers { permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 } } peers { permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 } }
I20260811 19:11:16.306671 7532 leader_election.cc:288] T 85cf312a02384676a6f6a60dd0d5dc9f P 5fa054465f054cdcaef05668dd6c801c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5943e4a12ecb475db4d131b5047b8f35 (127.3.61.129:46395), b60c32f9098646edaf34b986f782a645 (127.3.61.131:39303)
I20260811 19:11:16.315716 7617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:16.307190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.131:39303 (local address 127.3.61.130:60329)
0811 19:11:16.307966 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:16.307988 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:16.308156 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:16.309259 (+ 1103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:16.309265 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:16.309310 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:11:16.309638 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:16.309697 (+ 59us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:16.311114 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:16.311118 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:16.313040 (+ 1922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:16.313059 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:16.314695 (+ 1636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:16.314701 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:16.314920 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:16.314972 (+ 52us) client_negotiation.cc:769] Sending connection context
0811 19:11:16.315063 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 19:11:16.315309 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":656,"thread_start_us":531,"threads_started":1}
I20260811 19:11:16.315784 7619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:16.307555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:60329 (local address 127.3.61.131:39303)
0811 19:11:16.308900 (+ 1345us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:16.308906 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:16.308923 (+ 17us) server_negotiation.cc:407] Connection header received
0811 19:11:16.309001 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:16.309006 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:16.309094 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:16.309209 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:11:16.309875 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:16.310956 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:16.313239 (+ 2283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:16.314494 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:16.314671 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:16.314760 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:16.315278 (+ 518us) server_negotiation.cc:299] Negotiation successful
0811 19:11:16.315430 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1232,"thread_start_us":1118,"threads_started":1}
I20260811 19:11:16.316228 7511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:16.307351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:55613 (local address 127.3.61.129:46395)
0811 19:11:16.307581 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:16.307588 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:16.308866 (+ 1278us) server_negotiation.cc:407] Connection header received
0811 19:11:16.309020 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:16.309024 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:16.309094 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:16.309209 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:11:16.310422 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:16.311591 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:16.313775 (+ 2184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:16.314925 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:16.315108 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:16.315177 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:16.315624 (+ 447us) server_negotiation.cc:299] Negotiation successful
0811 19:11:16.315774 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 19:11:16.316777 7618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:16.307250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:46395 (local address 127.3.61.130:55613)
0811 19:11:16.308733 (+ 1483us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:16.308753 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:16.308894 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:16.309796 (+ 902us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:16.309801 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:16.309831 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:11:16.310230 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:16.310243 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:16.311750 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:16.311756 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:16.313606 (+ 1850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:16.313618 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:16.315182 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:16.315190 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:16.315389 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:16.315426 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 19:11:16.316275 (+ 849us) client_negotiation.cc:240] Negotiation successful
0811 19:11:16.316414 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1361,"mutex_wait_us":5,"thread_start_us":1184,"threads_started":1}
I20260811 19:11:16.316632 7388 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85cf312a02384676a6f6a60dd0d5dc9f" candidate_uuid: "5fa054465f054cdcaef05668dd6c801c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60c32f9098646edaf34b986f782a645" is_pre_election: true
I20260811 19:11:16.316967 7388 raft_consensus.cc:2454] T 85cf312a02384676a6f6a60dd0d5dc9f P b60c32f9098646edaf34b986f782a645 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5fa054465f054cdcaef05668dd6c801c in term 0.
I20260811 19:11:16.317579 6967 leader_election.cc:302] T 85cf312a02384676a6f6a60dd0d5dc9f P 5fa054465f054cdcaef05668dd6c801c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fa054465f054cdcaef05668dd6c801c, b60c32f9098646edaf34b986f782a645; no voters:
I20260811 19:11:16.318044 7532 raft_consensus.cc:2781] T 85cf312a02384676a6f6a60dd0d5dc9f P 5fa054465f054cdcaef05668dd6c801c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:11:16.317855 6830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85cf312a02384676a6f6a60dd0d5dc9f" candidate_uuid: "5fa054465f054cdcaef05668dd6c801c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5943e4a12ecb475db4d131b5047b8f35" is_pre_election: true
I20260811 19:11:16.318161 7532 raft_consensus.cc:482] T 85cf312a02384676a6f6a60dd0d5dc9f P 5fa054465f054cdcaef05668dd6c801c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:11:16.318259 7532 raft_consensus.cc:3037] T 85cf312a02384676a6f6a60dd0d5dc9f P 5fa054465f054cdcaef05668dd6c801c [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:11:16.318348 6830 raft_consensus.cc:2454] T 85cf312a02384676a6f6a60dd0d5dc9f P 5943e4a12ecb475db4d131b5047b8f35 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5fa054465f054cdcaef05668dd6c801c in term 0.
I20260811 19:11:16.321015 7532 raft_consensus.cc:504] T 85cf312a02384676a6f6a60dd0d5dc9f P 5fa054465f054cdcaef05668dd6c801c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 } } peers { permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 } } peers { permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 } }
I20260811 19:11:16.321647 7532 leader_election.cc:288] T 85cf312a02384676a6f6a60dd0d5dc9f P 5fa054465f054cdcaef05668dd6c801c [CANDIDATE]: Term 1 election: Requested vote from peers 5943e4a12ecb475db4d131b5047b8f35 (127.3.61.129:46395), b60c32f9098646edaf34b986f782a645 (127.3.61.131:39303)
I20260811 19:11:16.322201 6830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85cf312a02384676a6f6a60dd0d5dc9f" candidate_uuid: "5fa054465f054cdcaef05668dd6c801c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5943e4a12ecb475db4d131b5047b8f35"
I20260811 19:11:16.322355 7388 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85cf312a02384676a6f6a60dd0d5dc9f" candidate_uuid: "5fa054465f054cdcaef05668dd6c801c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60c32f9098646edaf34b986f782a645"
I20260811 19:11:16.322429 6830 raft_consensus.cc:3037] T 85cf312a02384676a6f6a60dd0d5dc9f P 5943e4a12ecb475db4d131b5047b8f35 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:11:16.322579 7388 raft_consensus.cc:3037] T 85cf312a02384676a6f6a60dd0d5dc9f P b60c32f9098646edaf34b986f782a645 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:11:16.324604 7388 raft_consensus.cc:2454] T 85cf312a02384676a6f6a60dd0d5dc9f P b60c32f9098646edaf34b986f782a645 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fa054465f054cdcaef05668dd6c801c in term 1.
I20260811 19:11:16.324987 6967 leader_election.cc:302] T 85cf312a02384676a6f6a60dd0d5dc9f P 5fa054465f054cdcaef05668dd6c801c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fa054465f054cdcaef05668dd6c801c, b60c32f9098646edaf34b986f782a645; no voters:
I20260811 19:11:16.325258 7532 raft_consensus.cc:2781] T 85cf312a02384676a6f6a60dd0d5dc9f P 5fa054465f054cdcaef05668dd6c801c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:11:16.325729 7532 raft_consensus.cc:686] T 85cf312a02384676a6f6a60dd0d5dc9f P 5fa054465f054cdcaef05668dd6c801c [term 1 LEADER]: Becoming Leader. State: Replica: 5fa054465f054cdcaef05668dd6c801c, State: Running, Role: LEADER
I20260811 19:11:16.326007 6830 raft_consensus.cc:2454] T 85cf312a02384676a6f6a60dd0d5dc9f P 5943e4a12ecb475db4d131b5047b8f35 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fa054465f054cdcaef05668dd6c801c in term 1.
I20260811 19:11:16.326182 7532 consensus_queue.cc:237] T 85cf312a02384676a6f6a60dd0d5dc9f P 5fa054465f054cdcaef05668dd6c801c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 } } peers { permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 } } peers { permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 } }
I20260811 19:11:16.330461 6530 catalog_manager.cc:5495] T 85cf312a02384676a6f6a60dd0d5dc9f P 5fa054465f054cdcaef05668dd6c801c reported cstate change: term changed from 0 to 1, leader changed from <none> to 5fa054465f054cdcaef05668dd6c801c (127.3.61.130). New cstate: current_term: 1 leader_uuid: "5fa054465f054cdcaef05668dd6c801c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 } health_report { overall_health: UNKNOWN } } }
I20260811 19:11:16.380334 7627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:16.374462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35296 (local address 127.3.61.130:38405)
0811 19:11:16.375043 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:16.375047 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:16.375066 (+ 19us) server_negotiation.cc:407] Connection header received
0811 19:11:16.375158 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:16.375162 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:16.375245 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:16.375397 (+ 152us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:11:16.375879 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:16.376665 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:16.378014 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:16.378431 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:16.378546 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:16.379016 (+ 470us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:11:16.379641 (+ 625us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:11:16.379775 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:16.379891 (+ 116us) server_negotiation.cc:299] Negotiation successful
0811 19:11:16.379989 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":315,"threads_started":1}
I20260811 19:11:16.400435 7511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:16.393567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:49747 (local address 127.3.61.129:46395)
0811 19:11:16.393807 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:16.393835 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:16.394548 (+ 713us) server_negotiation.cc:407] Connection header received
0811 19:11:16.394744 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:16.394749 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:16.394848 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:16.394964 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:11:16.395582 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:16.396433 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:16.398466 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:16.398917 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:16.399020 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:16.399614 (+ 594us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:11:16.399750 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:11:16.399821 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:16.400025 (+ 204us) server_negotiation.cc:299] Negotiation successful
0811 19:11:16.400108 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 19:11:16.400509 7633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:16.393531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:46395 (local address 127.3.61.130:49747)
0811 19:11:16.394348 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:16.394369 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:16.394530 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:16.394984 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:16.394989 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:16.395003 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 19:11:16.395327 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:16.395338 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:16.396571 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:16.396576 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:16.398275 (+ 1699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:16.398295 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:16.399113 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:16.399120 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:16.399303 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:16.399388 (+ 85us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 19:11:16.399849 (+ 461us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 19:11:16.399860 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 19:11:16.399941 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 19:11:16.400102 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"thread_start_us":404,"threads_started":1}
I20260811 19:11:16.411826 7388 raft_consensus.cc:1261] T 85cf312a02384676a6f6a60dd0d5dc9f P b60c32f9098646edaf34b986f782a645 [term 1 FOLLOWER]: Refusing update from remote peer 5fa054465f054cdcaef05668dd6c801c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:11:16.412324 6830 raft_consensus.cc:1261] T 85cf312a02384676a6f6a60dd0d5dc9f P 5943e4a12ecb475db4d131b5047b8f35 [term 1 FOLLOWER]: Refusing update from remote peer 5fa054465f054cdcaef05668dd6c801c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:11:16.412496 7621 consensus_queue.cc:1035] T 85cf312a02384676a6f6a60dd0d5dc9f P 5fa054465f054cdcaef05668dd6c801c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:11:16.413087 7532 consensus_queue.cc:1035] T 85cf312a02384676a6f6a60dd0d5dc9f P 5fa054465f054cdcaef05668dd6c801c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:11:16.420742 7581 mvcc.cc:205] Tried to move back new op lower bound from 7317403551376424960 to 7317403551036243968. Current Snapshot: MvccSnapshot[applied={T|T < 7317403551376424960}]
I20260811 19:11:16.456323 7599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:16.449739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.130:38405 (local address 127.3.61.129:49957)
0811 19:11:16.449964 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:16.449988 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:16.450136 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:16.450891 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:16.450895 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:16.450908 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 19:11:16.451146 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:16.451158 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:16.452665 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:16.452671 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:16.454019 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:16.454034 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:16.454999 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:16.455005 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:16.455196 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:16.455249 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 19:11:16.455747 (+ 498us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 19:11:16.455758 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 19:11:16.455836 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 19:11:16.455985 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 19:11:16.457188 7627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:16.450164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:49957 (local address 127.3.61.130:38405)
0811 19:11:16.450360 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:16.450366 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:16.450384 (+ 18us) server_negotiation.cc:407] Connection header received
0811 19:11:16.450454 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:16.450458 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:16.450557 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:16.450657 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:11:16.451390 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:16.452520 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:16.454169 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:16.454746 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:16.454869 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:16.455445 (+ 576us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:11:16.455610 (+ 165us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:11:16.455694 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:16.456667 (+ 973us) server_negotiation.cc:299] Negotiation successful
0811 19:11:16.456753 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
W20260811 19:11:16.458797 7582 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 19:11:18.087291 6533 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33540:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:11:18.087574 6533 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33540:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:11:18.093480 6533 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 404597c6971d4297830d78a22a801cac: Sending DeleteTablet for 3 replicas of tablet 85cf312a02384676a6f6a60dd0d5dc9f
I20260811 19:11:18.094681 7068 tablet_service.cc:1514] Processing DeleteTablet for tablet 85cf312a02384676a6f6a60dd0d5dc9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:11:18 UTC) from {username='kudu', principal='kudu/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:35282
I20260811 19:11:18.094671 6791 tablet_service.cc:1514] Processing DeleteTablet for tablet 85cf312a02384676a6f6a60dd0d5dc9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:11:18 UTC) from {username='kudu', principal='kudu/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:39448
I20260811 19:11:18.094681 7338 tablet_service.cc:1514] Processing DeleteTablet for tablet 85cf312a02384676a6f6a60dd0d5dc9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:11:18 UTC) from {username='kudu', principal='kudu/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:58612
I20260811 19:11:18.105621 7669 tablet_replica.cc:330] T 85cf312a02384676a6f6a60dd0d5dc9f P b60c32f9098646edaf34b986f782a645: stopping tablet replica
I20260811 19:11:18.105994 7669 raft_consensus.cc:2229] T 85cf312a02384676a6f6a60dd0d5dc9f P b60c32f9098646edaf34b986f782a645 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:11:18.106374 7669 raft_consensus.cc:2258] T 85cf312a02384676a6f6a60dd0d5dc9f P b60c32f9098646edaf34b986f782a645 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:11:18.107177 7672 tablet_replica.cc:330] T 85cf312a02384676a6f6a60dd0d5dc9f P 5943e4a12ecb475db4d131b5047b8f35: stopping tablet replica
I20260811 19:11:18.107487 7672 raft_consensus.cc:2229] T 85cf312a02384676a6f6a60dd0d5dc9f P 5943e4a12ecb475db4d131b5047b8f35 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:11:18.107820 7672 raft_consensus.cc:2258] T 85cf312a02384676a6f6a60dd0d5dc9f P 5943e4a12ecb475db4d131b5047b8f35 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:11:18.107972 7673 tablet_replica.cc:330] T 85cf312a02384676a6f6a60dd0d5dc9f P 5fa054465f054cdcaef05668dd6c801c: stopping tablet replica
I20260811 19:11:18.108580 7673 raft_consensus.cc:2229] T 85cf312a02384676a6f6a60dd0d5dc9f P 5fa054465f054cdcaef05668dd6c801c [term 1 LEADER]: Raft consensus shutting down.
I20260811 19:11:18.108879 7669 ts_tablet_manager.cc:1905] T 85cf312a02384676a6f6a60dd0d5dc9f P b60c32f9098646edaf34b986f782a645: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:11:18.109197 7673 raft_consensus.cc:2258] T 85cf312a02384676a6f6a60dd0d5dc9f P 5fa054465f054cdcaef05668dd6c801c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:11:18.110081 7672 ts_tablet_manager.cc:1905] T 85cf312a02384676a6f6a60dd0d5dc9f P 5943e4a12ecb475db4d131b5047b8f35: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:11:18.110860 7673 ts_tablet_manager.cc:1905] T 85cf312a02384676a6f6a60dd0d5dc9f P 5fa054465f054cdcaef05668dd6c801c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:11:18.117257 7673 ts_tablet_manager.cc:1918] T 85cf312a02384676a6f6a60dd0d5dc9f P 5fa054465f054cdcaef05668dd6c801c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 19:11:18.117545 7673 log.cc:1196] T 85cf312a02384676a6f6a60dd0d5dc9f P 5fa054465f054cdcaef05668dd6c801c: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-1/wal/wals/85cf312a02384676a6f6a60dd0d5dc9f
I20260811 19:11:18.117885 7669 ts_tablet_manager.cc:1918] T 85cf312a02384676a6f6a60dd0d5dc9f P b60c32f9098646edaf34b986f782a645: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 19:11:18.118072 7669 log.cc:1196] T 85cf312a02384676a6f6a60dd0d5dc9f P b60c32f9098646edaf34b986f782a645: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-2/wal/wals/85cf312a02384676a6f6a60dd0d5dc9f
I20260811 19:11:18.118170 7673 ts_tablet_manager.cc:1939] T 85cf312a02384676a6f6a60dd0d5dc9f P 5fa054465f054cdcaef05668dd6c801c: Deleting consensus metadata
I20260811 19:11:18.118605 7669 ts_tablet_manager.cc:1939] T 85cf312a02384676a6f6a60dd0d5dc9f P b60c32f9098646edaf34b986f782a645: Deleting consensus metadata
I20260811 19:11:18.118590 7672 ts_tablet_manager.cc:1918] T 85cf312a02384676a6f6a60dd0d5dc9f P 5943e4a12ecb475db4d131b5047b8f35: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 19:11:18.118782 7672 log.cc:1196] T 85cf312a02384676a6f6a60dd0d5dc9f P 5943e4a12ecb475db4d131b5047b8f35: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-0/wal/wals/85cf312a02384676a6f6a60dd0d5dc9f
I20260811 19:11:18.119568 6503 catalog_manager.cc:4841] TS 5fa054465f054cdcaef05668dd6c801c (127.3.61.130:38405): tablet 85cf312a02384676a6f6a60dd0d5dc9f (table test-table [id=14f09958ee1f407d9d3d6c2171b70148]) successfully deleted
I20260811 19:11:18.119730 7672 ts_tablet_manager.cc:1939] T 85cf312a02384676a6f6a60dd0d5dc9f P 5943e4a12ecb475db4d131b5047b8f35: Deleting consensus metadata
I20260811 19:11:18.119958 6504 catalog_manager.cc:4841] TS b60c32f9098646edaf34b986f782a645 (127.3.61.131:39303): tablet 85cf312a02384676a6f6a60dd0d5dc9f (table test-table [id=14f09958ee1f407d9d3d6c2171b70148]) successfully deleted
I20260811 19:11:18.120862 6503 catalog_manager.cc:4841] TS 5943e4a12ecb475db4d131b5047b8f35 (127.3.61.129:46395): tablet 85cf312a02384676a6f6a60dd0d5dc9f (table test-table [id=14f09958ee1f407d9d3d6c2171b70148]) successfully deleted
Aug 11 19:11:18 dist-test-slave-t7g5 krb5kdc[6453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475478, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 19:11:18 dist-test-slave-t7g5 krb5kdc[6453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475478, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:11:18.171473 7689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:18.152095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33556 (local address 127.3.61.190:33901)
0811 19:11:18.152969 (+ 874us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:18.152974 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:18.152998 (+ 24us) server_negotiation.cc:407] Connection header received
0811 19:11:18.153093 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:18.153097 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:18.153156 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:18.153241 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:18.155081 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:18.155901 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:18.158089 (+ 2188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:18.158653 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:18.158771 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:18.164293 (+ 5522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:18.164298 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:18.164303 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:18.164347 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:18.167718 (+ 3371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:18.168647 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:18.168652 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:18.168654 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:18.168873 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:18.169315 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:18.169319 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:18.169321 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:18.169753 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:18.170062 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:18.170605 (+ 543us) server_negotiation.cc:299] Negotiation successful
0811 19:11:18.170953 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":753,"thread_start_us":594,"threads_started":1}
I20260811 19:11:18.175611 6533 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33556:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 19:11:18.176384 6533 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:11:18.183913 6791 tablet_service.cc:1467] Processing CreateTablet for tablet 8f8e046947f64dae832311facc40b7e0 (DEFAULT_TABLE table=test-table [id=5212d63be848473599c33fab868082fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:11:18.184083 7068 tablet_service.cc:1467] Processing CreateTablet for tablet 8f8e046947f64dae832311facc40b7e0 (DEFAULT_TABLE table=test-table [id=5212d63be848473599c33fab868082fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:11:18.184398 6791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f8e046947f64dae832311facc40b7e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:11:18.184423 7338 tablet_service.cc:1467] Processing CreateTablet for tablet 8f8e046947f64dae832311facc40b7e0 (DEFAULT_TABLE table=test-table [id=5212d63be848473599c33fab868082fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:11:18.184544 7068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f8e046947f64dae832311facc40b7e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:11:18.184852 7338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f8e046947f64dae832311facc40b7e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:11:18.200212 7697 tablet_bootstrap.cc:492] T 8f8e046947f64dae832311facc40b7e0 P 5fa054465f054cdcaef05668dd6c801c: Bootstrap starting.
I20260811 19:11:18.200603 7695 tablet_bootstrap.cc:492] T 8f8e046947f64dae832311facc40b7e0 P b60c32f9098646edaf34b986f782a645: Bootstrap starting.
I20260811 19:11:18.203248 7695 tablet_bootstrap.cc:654] T 8f8e046947f64dae832311facc40b7e0 P b60c32f9098646edaf34b986f782a645: Neither blocks nor log segments found. Creating new log.
I20260811 19:11:18.203433 7697 tablet_bootstrap.cc:654] T 8f8e046947f64dae832311facc40b7e0 P 5fa054465f054cdcaef05668dd6c801c: Neither blocks nor log segments found. Creating new log.
I20260811 19:11:18.205426 7693 tablet_bootstrap.cc:492] T 8f8e046947f64dae832311facc40b7e0 P 5943e4a12ecb475db4d131b5047b8f35: Bootstrap starting.
I20260811 19:11:18.207166 7697 tablet_bootstrap.cc:492] T 8f8e046947f64dae832311facc40b7e0 P 5fa054465f054cdcaef05668dd6c801c: No bootstrap required, opened a new log
I20260811 19:11:18.207373 7697 ts_tablet_manager.cc:1397] T 8f8e046947f64dae832311facc40b7e0 P 5fa054465f054cdcaef05668dd6c801c: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 19:11:18.207700 7693 tablet_bootstrap.cc:654] T 8f8e046947f64dae832311facc40b7e0 P 5943e4a12ecb475db4d131b5047b8f35: Neither blocks nor log segments found. Creating new log.
I20260811 19:11:18.207743 7695 tablet_bootstrap.cc:492] T 8f8e046947f64dae832311facc40b7e0 P b60c32f9098646edaf34b986f782a645: No bootstrap required, opened a new log
I20260811 19:11:18.208094 7695 ts_tablet_manager.cc:1397] T 8f8e046947f64dae832311facc40b7e0 P b60c32f9098646edaf34b986f782a645: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260811 19:11:18.208391 7697 raft_consensus.cc:348] T 8f8e046947f64dae832311facc40b7e0 P 5fa054465f054cdcaef05668dd6c801c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 } } peers { permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 } } peers { permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 } }
I20260811 19:11:18.208654 7697 raft_consensus.cc:374] T 8f8e046947f64dae832311facc40b7e0 P 5fa054465f054cdcaef05668dd6c801c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:11:18.208789 7697 raft_consensus.cc:729] T 8f8e046947f64dae832311facc40b7e0 P 5fa054465f054cdcaef05668dd6c801c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fa054465f054cdcaef05668dd6c801c, State: Initialized, Role: FOLLOWER
I20260811 19:11:18.209121 7697 consensus_queue.cc:260] T 8f8e046947f64dae832311facc40b7e0 P 5fa054465f054cdcaef05668dd6c801c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 } } peers { permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 } } peers { permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 } }
I20260811 19:11:18.209152 7695 raft_consensus.cc:348] T 8f8e046947f64dae832311facc40b7e0 P b60c32f9098646edaf34b986f782a645 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 } } peers { permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 } } peers { permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 } }
I20260811 19:11:18.209411 7695 raft_consensus.cc:374] T 8f8e046947f64dae832311facc40b7e0 P b60c32f9098646edaf34b986f782a645 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:11:18.209543 7695 raft_consensus.cc:729] T 8f8e046947f64dae832311facc40b7e0 P b60c32f9098646edaf34b986f782a645 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60c32f9098646edaf34b986f782a645, State: Initialized, Role: FOLLOWER
I20260811 19:11:18.209944 7697 ts_tablet_manager.cc:1428] T 8f8e046947f64dae832311facc40b7e0 P 5fa054465f054cdcaef05668dd6c801c: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 19:11:18.209848 7695 consensus_queue.cc:260] T 8f8e046947f64dae832311facc40b7e0 P b60c32f9098646edaf34b986f782a645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 } } peers { permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 } } peers { permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 } }
I20260811 19:11:18.210906 7693 tablet_bootstrap.cc:492] T 8f8e046947f64dae832311facc40b7e0 P 5943e4a12ecb475db4d131b5047b8f35: No bootstrap required, opened a new log
I20260811 19:11:18.211026 7695 ts_tablet_manager.cc:1428] T 8f8e046947f64dae832311facc40b7e0 P b60c32f9098646edaf34b986f782a645: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 19:11:18.211102 7693 ts_tablet_manager.cc:1397] T 8f8e046947f64dae832311facc40b7e0 P 5943e4a12ecb475db4d131b5047b8f35: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 19:11:18.212373 7693 raft_consensus.cc:348] T 8f8e046947f64dae832311facc40b7e0 P 5943e4a12ecb475db4d131b5047b8f35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 } } peers { permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 } } peers { permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 } }
I20260811 19:11:18.212707 7693 raft_consensus.cc:374] T 8f8e046947f64dae832311facc40b7e0 P 5943e4a12ecb475db4d131b5047b8f35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:11:18.212852 7693 raft_consensus.cc:729] T 8f8e046947f64dae832311facc40b7e0 P 5943e4a12ecb475db4d131b5047b8f35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5943e4a12ecb475db4d131b5047b8f35, State: Initialized, Role: FOLLOWER
I20260811 19:11:18.213160 7693 consensus_queue.cc:260] T 8f8e046947f64dae832311facc40b7e0 P 5943e4a12ecb475db4d131b5047b8f35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 } } peers { permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 } } peers { permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 } }
I20260811 19:11:18.213824 7693 ts_tablet_manager.cc:1428] T 8f8e046947f64dae832311facc40b7e0 P 5943e4a12ecb475db4d131b5047b8f35: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260811 19:11:18.250532 7643 raft_consensus.cc:482] T 8f8e046947f64dae832311facc40b7e0 P 5943e4a12ecb475db4d131b5047b8f35 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:11:18.250890 7643 raft_consensus.cc:504] T 8f8e046947f64dae832311facc40b7e0 P 5943e4a12ecb475db4d131b5047b8f35 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 } } peers { permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 } } peers { permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 } }
I20260811 19:11:18.251724 7643 leader_election.cc:288] T 8f8e046947f64dae832311facc40b7e0 P 5943e4a12ecb475db4d131b5047b8f35 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5fa054465f054cdcaef05668dd6c801c (127.3.61.130:38405), b60c32f9098646edaf34b986f782a645 (127.3.61.131:39303)
I20260811 19:11:18.252434 7388 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f8e046947f64dae832311facc40b7e0" candidate_uuid: "5943e4a12ecb475db4d131b5047b8f35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60c32f9098646edaf34b986f782a645" is_pre_election: true
I20260811 19:11:18.252425 7112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f8e046947f64dae832311facc40b7e0" candidate_uuid: "5943e4a12ecb475db4d131b5047b8f35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fa054465f054cdcaef05668dd6c801c" is_pre_election: true
I20260811 19:11:18.252727 7112 raft_consensus.cc:2454] T 8f8e046947f64dae832311facc40b7e0 P 5fa054465f054cdcaef05668dd6c801c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5943e4a12ecb475db4d131b5047b8f35 in term 0.
I20260811 19:11:18.252729 7388 raft_consensus.cc:2454] T 8f8e046947f64dae832311facc40b7e0 P b60c32f9098646edaf34b986f782a645 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5943e4a12ecb475db4d131b5047b8f35 in term 0.
I20260811 19:11:18.253437 6691 leader_election.cc:302] T 8f8e046947f64dae832311facc40b7e0 P 5943e4a12ecb475db4d131b5047b8f35 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 5943e4a12ecb475db4d131b5047b8f35, 5fa054465f054cdcaef05668dd6c801c, b60c32f9098646edaf34b986f782a645; no voters:
I20260811 19:11:18.253791 7643 raft_consensus.cc:2781] T 8f8e046947f64dae832311facc40b7e0 P 5943e4a12ecb475db4d131b5047b8f35 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:11:18.253934 7643 raft_consensus.cc:482] T 8f8e046947f64dae832311facc40b7e0 P 5943e4a12ecb475db4d131b5047b8f35 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:11:18.254031 7643 raft_consensus.cc:3037] T 8f8e046947f64dae832311facc40b7e0 P 5943e4a12ecb475db4d131b5047b8f35 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:11:18.257923 7643 raft_consensus.cc:504] T 8f8e046947f64dae832311facc40b7e0 P 5943e4a12ecb475db4d131b5047b8f35 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 } } peers { permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 } } peers { permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 } }
I20260811 19:11:18.258797 7643 leader_election.cc:288] T 8f8e046947f64dae832311facc40b7e0 P 5943e4a12ecb475db4d131b5047b8f35 [CANDIDATE]: Term 1 election: Requested vote from peers 5fa054465f054cdcaef05668dd6c801c (127.3.61.130:38405), b60c32f9098646edaf34b986f782a645 (127.3.61.131:39303)
I20260811 19:11:18.259462 7112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f8e046947f64dae832311facc40b7e0" candidate_uuid: "5943e4a12ecb475db4d131b5047b8f35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fa054465f054cdcaef05668dd6c801c"
I20260811 19:11:18.259510 7388 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f8e046947f64dae832311facc40b7e0" candidate_uuid: "5943e4a12ecb475db4d131b5047b8f35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60c32f9098646edaf34b986f782a645"
I20260811 19:11:18.259717 7112 raft_consensus.cc:3037] T 8f8e046947f64dae832311facc40b7e0 P 5fa054465f054cdcaef05668dd6c801c [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:11:18.259748 7388 raft_consensus.cc:3037] T 8f8e046947f64dae832311facc40b7e0 P b60c32f9098646edaf34b986f782a645 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:11:18.262427 7112 raft_consensus.cc:2454] T 8f8e046947f64dae832311facc40b7e0 P 5fa054465f054cdcaef05668dd6c801c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5943e4a12ecb475db4d131b5047b8f35 in term 1.
I20260811 19:11:18.262920 7388 raft_consensus.cc:2454] T 8f8e046947f64dae832311facc40b7e0 P b60c32f9098646edaf34b986f782a645 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5943e4a12ecb475db4d131b5047b8f35 in term 1.
I20260811 19:11:18.263018 6689 leader_election.cc:302] T 8f8e046947f64dae832311facc40b7e0 P 5943e4a12ecb475db4d131b5047b8f35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5943e4a12ecb475db4d131b5047b8f35, 5fa054465f054cdcaef05668dd6c801c; no voters:
I20260811 19:11:18.263433 7643 raft_consensus.cc:2781] T 8f8e046947f64dae832311facc40b7e0 P 5943e4a12ecb475db4d131b5047b8f35 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:11:18.263621 7643 raft_consensus.cc:686] T 8f8e046947f64dae832311facc40b7e0 P 5943e4a12ecb475db4d131b5047b8f35 [term 1 LEADER]: Becoming Leader. State: Replica: 5943e4a12ecb475db4d131b5047b8f35, State: Running, Role: LEADER
I20260811 19:11:18.264001 7643 consensus_queue.cc:237] T 8f8e046947f64dae832311facc40b7e0 P 5943e4a12ecb475db4d131b5047b8f35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 } } peers { permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 } } peers { permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 } }
I20260811 19:11:18.266981 6536 catalog_manager.cc:5495] T 8f8e046947f64dae832311facc40b7e0 P 5943e4a12ecb475db4d131b5047b8f35 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5943e4a12ecb475db4d131b5047b8f35 (127.3.61.129). New cstate: current_term: 1 leader_uuid: "5943e4a12ecb475db4d131b5047b8f35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 } health_report { overall_health: UNKNOWN } } }
I20260811 19:11:18.305922 7711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:18.297311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39454 (local address 127.3.61.129:46395)
0811 19:11:18.298228 (+ 917us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:18.298232 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:18.298258 (+ 26us) server_negotiation.cc:407] Connection header received
0811 19:11:18.298316 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:18.298319 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:18.298384 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:18.298471 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:11:18.299131 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:18.300539 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:18.303039 (+ 2500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:18.303845 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:18.303986 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:18.304810 (+ 824us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:11:18.304980 (+ 170us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:11:18.305069 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:18.305308 (+ 239us) server_negotiation.cc:299] Negotiation successful
0811 19:11:18.305388 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":781,"thread_start_us":576,"threads_started":1}
I20260811 19:11:18.320879 7388 raft_consensus.cc:1261] T 8f8e046947f64dae832311facc40b7e0 P b60c32f9098646edaf34b986f782a645 [term 1 FOLLOWER]: Refusing update from remote peer 5943e4a12ecb475db4d131b5047b8f35: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:11:18.321065 7112 raft_consensus.cc:1261] T 8f8e046947f64dae832311facc40b7e0 P 5fa054465f054cdcaef05668dd6c801c [term 1 FOLLOWER]: Refusing update from remote peer 5943e4a12ecb475db4d131b5047b8f35: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:11:18.321970 7640 consensus_queue.cc:1035] T 8f8e046947f64dae832311facc40b7e0 P 5943e4a12ecb475db4d131b5047b8f35 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:11:18.322505 7643 consensus_queue.cc:1035] T 8f8e046947f64dae832311facc40b7e0 P 5943e4a12ecb475db4d131b5047b8f35 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:11:18.329840 7647 mvcc.cc:205] Tried to move back new op lower bound from 7317403559195443200 to 7317403558971990016. Current Snapshot: MvccSnapshot[applied={T|T < 7317403559195443200}]
I20260811 19:11:18.372679 6536 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33556:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:11:18.373258 6536 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33556:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:11:18.378141 6536 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 404597c6971d4297830d78a22a801cac: Sending DeleteTablet for 3 replicas of tablet 8f8e046947f64dae832311facc40b7e0
I20260811 19:11:18.379284 7068 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f8e046947f64dae832311facc40b7e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:11:18 UTC) from {username='kudu', principal='kudu/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:35282
I20260811 19:11:18.379321 7338 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f8e046947f64dae832311facc40b7e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:11:18 UTC) from {username='kudu', principal='kudu/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:58612
I20260811 19:11:18.379635 7673 tablet_replica.cc:330] T 8f8e046947f64dae832311facc40b7e0 P 5fa054465f054cdcaef05668dd6c801c: stopping tablet replica
I20260811 19:11:18.379740 7669 tablet_replica.cc:330] T 8f8e046947f64dae832311facc40b7e0 P b60c32f9098646edaf34b986f782a645: stopping tablet replica
I20260811 19:11:18.379884 7673 raft_consensus.cc:2229] T 8f8e046947f64dae832311facc40b7e0 P 5fa054465f054cdcaef05668dd6c801c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:11:18.379978 7669 raft_consensus.cc:2229] T 8f8e046947f64dae832311facc40b7e0 P b60c32f9098646edaf34b986f782a645 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:11:18.379987 6791 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f8e046947f64dae832311facc40b7e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:11:18 UTC) from {username='kudu', principal='kudu/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:39448
I20260811 19:11:18.380148 7673 raft_consensus.cc:2258] T 8f8e046947f64dae832311facc40b7e0 P 5fa054465f054cdcaef05668dd6c801c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:11:18.380208 7669 raft_consensus.cc:2258] T 8f8e046947f64dae832311facc40b7e0 P b60c32f9098646edaf34b986f782a645 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:11:18.380499 7672 tablet_replica.cc:330] T 8f8e046947f64dae832311facc40b7e0 P 5943e4a12ecb475db4d131b5047b8f35: stopping tablet replica
I20260811 19:11:18.380280 6536 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33556:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 19:11:18.380740 7672 raft_consensus.cc:2229] T 8f8e046947f64dae832311facc40b7e0 P 5943e4a12ecb475db4d131b5047b8f35 [term 1 LEADER]: Raft consensus shutting down.
W20260811 19:11:18.380987 6536 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:11:18.381112 7672 raft_consensus.cc:2258] T 8f8e046947f64dae832311facc40b7e0 P 5943e4a12ecb475db4d131b5047b8f35 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:11:18.381479 7669 ts_tablet_manager.cc:1905] T 8f8e046947f64dae832311facc40b7e0 P b60c32f9098646edaf34b986f782a645: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:11:18.382252 7672 ts_tablet_manager.cc:1905] T 8f8e046947f64dae832311facc40b7e0 P 5943e4a12ecb475db4d131b5047b8f35: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:11:18.382735 7673 ts_tablet_manager.cc:1905] T 8f8e046947f64dae832311facc40b7e0 P 5fa054465f054cdcaef05668dd6c801c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:11:18.389001 7669 ts_tablet_manager.cc:1918] T 8f8e046947f64dae832311facc40b7e0 P b60c32f9098646edaf34b986f782a645: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 19:11:18.389549 7672 ts_tablet_manager.cc:1918] T 8f8e046947f64dae832311facc40b7e0 P 5943e4a12ecb475db4d131b5047b8f35: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 19:11:18.389845 7672 log.cc:1196] T 8f8e046947f64dae832311facc40b7e0 P 5943e4a12ecb475db4d131b5047b8f35: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-0/wal/wals/8f8e046947f64dae832311facc40b7e0
I20260811 19:11:18.390115 7338 tablet_service.cc:1467] Processing CreateTablet for tablet 6c1befe1fbee42a1b98534838bfafd0e (DEFAULT_TABLE table=test-table [id=cfcde4dcded84509804717f30aacb4ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:11:18.390276 7068 tablet_service.cc:1467] Processing CreateTablet for tablet 6c1befe1fbee42a1b98534838bfafd0e (DEFAULT_TABLE table=test-table [id=cfcde4dcded84509804717f30aacb4ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:11:18.390585 7338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c1befe1fbee42a1b98534838bfafd0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:11:18.390724 7068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c1befe1fbee42a1b98534838bfafd0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:11:18.389281 7669 log.cc:1196] T 8f8e046947f64dae832311facc40b7e0 P b60c32f9098646edaf34b986f782a645: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-2/wal/wals/8f8e046947f64dae832311facc40b7e0
I20260811 19:11:18.392030 7669 ts_tablet_manager.cc:1939] T 8f8e046947f64dae832311facc40b7e0 P b60c32f9098646edaf34b986f782a645: Deleting consensus metadata
I20260811 19:11:18.392648 7673 ts_tablet_manager.cc:1918] T 8f8e046947f64dae832311facc40b7e0 P 5fa054465f054cdcaef05668dd6c801c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 19:11:18.392942 7673 log.cc:1196] T 8f8e046947f64dae832311facc40b7e0 P 5fa054465f054cdcaef05668dd6c801c: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-1/wal/wals/8f8e046947f64dae832311facc40b7e0
I20260811 19:11:18.393690 6504 catalog_manager.cc:4841] TS b60c32f9098646edaf34b986f782a645 (127.3.61.131:39303): tablet 8f8e046947f64dae832311facc40b7e0 (table test-table [id=5212d63be848473599c33fab868082fb]) successfully deleted
I20260811 19:11:18.394860 7673 ts_tablet_manager.cc:1939] T 8f8e046947f64dae832311facc40b7e0 P 5fa054465f054cdcaef05668dd6c801c: Deleting consensus metadata
I20260811 19:11:18.390857 6791 tablet_service.cc:1467] Processing CreateTablet for tablet 6c1befe1fbee42a1b98534838bfafd0e (DEFAULT_TABLE table=test-table [id=cfcde4dcded84509804717f30aacb4ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:11:18.395429 6791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c1befe1fbee42a1b98534838bfafd0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:11:18.395552 7672 ts_tablet_manager.cc:1939] T 8f8e046947f64dae832311facc40b7e0 P 5943e4a12ecb475db4d131b5047b8f35: Deleting consensus metadata
I20260811 19:11:18.397989 6503 catalog_manager.cc:4841] TS 5943e4a12ecb475db4d131b5047b8f35 (127.3.61.129:46395): tablet 8f8e046947f64dae832311facc40b7e0 (table test-table [id=5212d63be848473599c33fab868082fb]) successfully deleted
I20260811 19:11:18.398178 7697 tablet_bootstrap.cc:492] T 6c1befe1fbee42a1b98534838bfafd0e P 5fa054465f054cdcaef05668dd6c801c: Bootstrap starting.
I20260811 19:11:18.398202 7695 tablet_bootstrap.cc:492] T 6c1befe1fbee42a1b98534838bfafd0e P b60c32f9098646edaf34b986f782a645: Bootstrap starting.
I20260811 19:11:18.400331 7697 tablet_bootstrap.cc:654] T 6c1befe1fbee42a1b98534838bfafd0e P 5fa054465f054cdcaef05668dd6c801c: Neither blocks nor log segments found. Creating new log.
I20260811 19:11:18.400331 7695 tablet_bootstrap.cc:654] T 6c1befe1fbee42a1b98534838bfafd0e P b60c32f9098646edaf34b986f782a645: Neither blocks nor log segments found. Creating new log.
I20260811 19:11:18.402576 6503 catalog_manager.cc:4841] TS 5fa054465f054cdcaef05668dd6c801c (127.3.61.130:38405): tablet 8f8e046947f64dae832311facc40b7e0 (table test-table [id=5212d63be848473599c33fab868082fb]) successfully deleted
I20260811 19:11:18.404053 7693 tablet_bootstrap.cc:492] T 6c1befe1fbee42a1b98534838bfafd0e P 5943e4a12ecb475db4d131b5047b8f35: Bootstrap starting.
I20260811 19:11:18.404145 7695 tablet_bootstrap.cc:492] T 6c1befe1fbee42a1b98534838bfafd0e P b60c32f9098646edaf34b986f782a645: No bootstrap required, opened a new log
I20260811 19:11:18.404428 7695 ts_tablet_manager.cc:1397] T 6c1befe1fbee42a1b98534838bfafd0e P b60c32f9098646edaf34b986f782a645: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260811 19:11:18.405488 7695 raft_consensus.cc:348] T 6c1befe1fbee42a1b98534838bfafd0e P b60c32f9098646edaf34b986f782a645 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 } } peers { permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 } } peers { permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 } }
I20260811 19:11:18.405876 7695 raft_consensus.cc:374] T 6c1befe1fbee42a1b98534838bfafd0e P b60c32f9098646edaf34b986f782a645 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:11:18.406010 7695 raft_consensus.cc:729] T 6c1befe1fbee42a1b98534838bfafd0e P b60c32f9098646edaf34b986f782a645 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60c32f9098646edaf34b986f782a645, State: Initialized, Role: FOLLOWER
I20260811 19:11:18.406229 7697 tablet_bootstrap.cc:492] T 6c1befe1fbee42a1b98534838bfafd0e P 5fa054465f054cdcaef05668dd6c801c: No bootstrap required, opened a new log
I20260811 19:11:18.406296 7693 tablet_bootstrap.cc:654] T 6c1befe1fbee42a1b98534838bfafd0e P 5943e4a12ecb475db4d131b5047b8f35: Neither blocks nor log segments found. Creating new log.
I20260811 19:11:18.406445 7697 ts_tablet_manager.cc:1397] T 6c1befe1fbee42a1b98534838bfafd0e P 5fa054465f054cdcaef05668dd6c801c: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260811 19:11:18.406344 7695 consensus_queue.cc:260] T 6c1befe1fbee42a1b98534838bfafd0e P b60c32f9098646edaf34b986f782a645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 } } peers { permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 } } peers { permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 } }
I20260811 19:11:18.406927 7695 ts_tablet_manager.cc:1428] T 6c1befe1fbee42a1b98534838bfafd0e P b60c32f9098646edaf34b986f782a645: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 19:11:18.407279 7697 raft_consensus.cc:348] T 6c1befe1fbee42a1b98534838bfafd0e P 5fa054465f054cdcaef05668dd6c801c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 } } peers { permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 } } peers { permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 } }
I20260811 19:11:18.407594 7697 raft_consensus.cc:374] T 6c1befe1fbee42a1b98534838bfafd0e P 5fa054465f054cdcaef05668dd6c801c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:11:18.407704 7697 raft_consensus.cc:729] T 6c1befe1fbee42a1b98534838bfafd0e P 5fa054465f054cdcaef05668dd6c801c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fa054465f054cdcaef05668dd6c801c, State: Initialized, Role: FOLLOWER
I20260811 19:11:18.407984 7697 consensus_queue.cc:260] T 6c1befe1fbee42a1b98534838bfafd0e P 5fa054465f054cdcaef05668dd6c801c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 } } peers { permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 } } peers { permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 } }
I20260811 19:11:18.408739 7697 ts_tablet_manager.cc:1428] T 6c1befe1fbee42a1b98534838bfafd0e P 5fa054465f054cdcaef05668dd6c801c: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 19:11:18.409638 7693 tablet_bootstrap.cc:492] T 6c1befe1fbee42a1b98534838bfafd0e P 5943e4a12ecb475db4d131b5047b8f35: No bootstrap required, opened a new log
I20260811 19:11:18.409955 7693 ts_tablet_manager.cc:1397] T 6c1befe1fbee42a1b98534838bfafd0e P 5943e4a12ecb475db4d131b5047b8f35: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260811 19:11:18.410987 7693 raft_consensus.cc:348] T 6c1befe1fbee42a1b98534838bfafd0e P 5943e4a12ecb475db4d131b5047b8f35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 } } peers { permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 } } peers { permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 } }
I20260811 19:11:18.411227 7693 raft_consensus.cc:374] T 6c1befe1fbee42a1b98534838bfafd0e P 5943e4a12ecb475db4d131b5047b8f35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:11:18.411340 7693 raft_consensus.cc:729] T 6c1befe1fbee42a1b98534838bfafd0e P 5943e4a12ecb475db4d131b5047b8f35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5943e4a12ecb475db4d131b5047b8f35, State: Initialized, Role: FOLLOWER
I20260811 19:11:18.411640 7693 consensus_queue.cc:260] T 6c1befe1fbee42a1b98534838bfafd0e P 5943e4a12ecb475db4d131b5047b8f35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 } } peers { permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 } } peers { permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 } }
I20260811 19:11:18.412272 7693 ts_tablet_manager.cc:1428] T 6c1befe1fbee42a1b98534838bfafd0e P 5943e4a12ecb475db4d131b5047b8f35: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 19:11:18.475152 7705 raft_consensus.cc:482] T 6c1befe1fbee42a1b98534838bfafd0e P b60c32f9098646edaf34b986f782a645 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:11:18.475484 7705 raft_consensus.cc:504] T 6c1befe1fbee42a1b98534838bfafd0e P b60c32f9098646edaf34b986f782a645 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 } } peers { permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 } } peers { permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 } }
I20260811 19:11:18.478015 7705 leader_election.cc:288] T 6c1befe1fbee42a1b98534838bfafd0e P b60c32f9098646edaf34b986f782a645 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5943e4a12ecb475db4d131b5047b8f35 (127.3.61.129:46395), 5fa054465f054cdcaef05668dd6c801c (127.3.61.130:38405)
I20260811 19:11:18.487077 7711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:18.478786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:41675 (local address 127.3.61.129:46395)
0811 19:11:18.478978 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:18.478983 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:18.479766 (+ 783us) server_negotiation.cc:407] Connection header received
0811 19:11:18.480020 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:18.480026 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:18.480119 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:18.480218 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:11:18.481509 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:18.482636 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:18.485048 (+ 2412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:18.486075 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:18.486219 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:18.486279 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:18.486720 (+ 441us) server_negotiation.cc:299] Negotiation successful
0811 19:11:18.486831 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 19:11:18.487349 7725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:18.478549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:46395 (local address 127.3.61.131:41675)
0811 19:11:18.479563 (+ 1014us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:18.479587 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:18.479820 (+ 233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:18.480477 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:18.480483 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:18.480523 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:11:18.481263 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:18.481278 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:18.482836 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:18.482840 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:18.484850 (+ 2010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:18.484863 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:18.486318 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:18.486324 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:18.486521 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:18.486562 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 19:11:18.486673 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 19:11:18.486818 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":836,"mutex_wait_us":1,"thread_start_us":480,"threads_started":1}
I20260811 19:11:18.488320 6830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c1befe1fbee42a1b98534838bfafd0e" candidate_uuid: "b60c32f9098646edaf34b986f782a645" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5943e4a12ecb475db4d131b5047b8f35" is_pre_election: true
I20260811 19:11:18.488654 6830 raft_consensus.cc:2454] T 6c1befe1fbee42a1b98534838bfafd0e P 5943e4a12ecb475db4d131b5047b8f35 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60c32f9098646edaf34b986f782a645 in term 0.
I20260811 19:11:18.489326 7232 leader_election.cc:302] T 6c1befe1fbee42a1b98534838bfafd0e P b60c32f9098646edaf34b986f782a645 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5943e4a12ecb475db4d131b5047b8f35, b60c32f9098646edaf34b986f782a645; no voters:
I20260811 19:11:18.489763 7729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:18.480070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:34985 (local address 127.3.61.130:38405)
0811 19:11:18.482466 (+ 2396us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:18.482473 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:18.482512 (+ 39us) server_negotiation.cc:407] Connection header received
0811 19:11:18.482695 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:18.482705 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:18.482808 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:18.482955 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:11:18.484045 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:18.484975 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:18.487401 (+ 2426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:18.488559 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:18.488788 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:18.488899 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:18.489233 (+ 334us) server_negotiation.cc:299] Negotiation successful
0811 19:11:18.489394 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2199,"thread_start_us":1935,"threads_started":1}
I20260811 19:11:18.490002 7727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:18.479895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.130:38405 (local address 127.3.61.131:34985)
0811 19:11:18.481324 (+ 1429us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:18.481350 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:18.481535 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:18.482992 (+ 1457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:18.482997 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:18.483029 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:11:18.483741 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:18.483760 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:18.485137 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:18.485141 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:18.487159 (+ 2018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:18.487178 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:18.488823 (+ 1645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:18.488831 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:18.489025 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:18.489059 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 19:11:18.489161 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 19:11:18.489329 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":994,"mutex_wait_us":22,"thread_start_us":596,"threads_started":1}
I20260811 19:11:18.489986 7705 raft_consensus.cc:2781] T 6c1befe1fbee42a1b98534838bfafd0e P b60c32f9098646edaf34b986f782a645 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:11:18.490211 7705 raft_consensus.cc:482] T 6c1befe1fbee42a1b98534838bfafd0e P b60c32f9098646edaf34b986f782a645 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:11:18.490334 7705 raft_consensus.cc:3037] T 6c1befe1fbee42a1b98534838bfafd0e P b60c32f9098646edaf34b986f782a645 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:11:18.492851 7705 raft_consensus.cc:504] T 6c1befe1fbee42a1b98534838bfafd0e P b60c32f9098646edaf34b986f782a645 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 } } peers { permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 } } peers { permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 } }
I20260811 19:11:18.493712 7705 leader_election.cc:288] T 6c1befe1fbee42a1b98534838bfafd0e P b60c32f9098646edaf34b986f782a645 [CANDIDATE]: Term 1 election: Requested vote from peers 5943e4a12ecb475db4d131b5047b8f35 (127.3.61.129:46395), 5fa054465f054cdcaef05668dd6c801c (127.3.61.130:38405)
I20260811 19:11:18.494297 7112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c1befe1fbee42a1b98534838bfafd0e" candidate_uuid: "b60c32f9098646edaf34b986f782a645" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fa054465f054cdcaef05668dd6c801c" is_pre_election: true
I20260811 19:11:18.494680 7112 raft_consensus.cc:2454] T 6c1befe1fbee42a1b98534838bfafd0e P 5fa054465f054cdcaef05668dd6c801c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b60c32f9098646edaf34b986f782a645 in term 0.
I20260811 19:11:18.494704 6830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c1befe1fbee42a1b98534838bfafd0e" candidate_uuid: "b60c32f9098646edaf34b986f782a645" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5943e4a12ecb475db4d131b5047b8f35"
I20260811 19:11:18.494730 7114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c1befe1fbee42a1b98534838bfafd0e" candidate_uuid: "b60c32f9098646edaf34b986f782a645" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fa054465f054cdcaef05668dd6c801c"
I20260811 19:11:18.494930 6830 raft_consensus.cc:3037] T 6c1befe1fbee42a1b98534838bfafd0e P 5943e4a12ecb475db4d131b5047b8f35 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:11:18.495075 7114 raft_consensus.cc:3037] T 6c1befe1fbee42a1b98534838bfafd0e P 5fa054465f054cdcaef05668dd6c801c [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:11:18.498116 6830 raft_consensus.cc:2454] T 6c1befe1fbee42a1b98534838bfafd0e P 5943e4a12ecb475db4d131b5047b8f35 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60c32f9098646edaf34b986f782a645 in term 1.
I20260811 19:11:18.498345 7114 raft_consensus.cc:2454] T 6c1befe1fbee42a1b98534838bfafd0e P 5fa054465f054cdcaef05668dd6c801c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b60c32f9098646edaf34b986f782a645 in term 1.
I20260811 19:11:18.498685 7232 leader_election.cc:302] T 6c1befe1fbee42a1b98534838bfafd0e P b60c32f9098646edaf34b986f782a645 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5943e4a12ecb475db4d131b5047b8f35, b60c32f9098646edaf34b986f782a645; no voters:
I20260811 19:11:18.499073 7705 raft_consensus.cc:2781] T 6c1befe1fbee42a1b98534838bfafd0e P b60c32f9098646edaf34b986f782a645 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:11:18.500149 7705 raft_consensus.cc:686] T 6c1befe1fbee42a1b98534838bfafd0e P b60c32f9098646edaf34b986f782a645 [term 1 LEADER]: Becoming Leader. State: Replica: b60c32f9098646edaf34b986f782a645, State: Running, Role: LEADER
I20260811 19:11:18.500862 7705 consensus_queue.cc:237] T 6c1befe1fbee42a1b98534838bfafd0e P b60c32f9098646edaf34b986f782a645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 } } peers { permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 } } peers { permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 } }
I20260811 19:11:18.505254 6530 catalog_manager.cc:5495] T 6c1befe1fbee42a1b98534838bfafd0e P b60c32f9098646edaf34b986f782a645 reported cstate change: term changed from 0 to 1, leader changed from <none> to b60c32f9098646edaf34b986f782a645 (127.3.61.131). New cstate: current_term: 1 leader_uuid: "b60c32f9098646edaf34b986f782a645" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b60c32f9098646edaf34b986f782a645" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39303 } health_report { overall_health: HEALTHY } } }
I20260811 19:11:18.550992 7733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:18.543641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58616 (local address 127.3.61.131:39303)
0811 19:11:18.544558 (+ 917us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:18.544562 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:18.544590 (+ 28us) server_negotiation.cc:407] Connection header received
0811 19:11:18.544690 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:18.544693 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:18.544776 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:18.544964 (+ 188us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:11:18.545518 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:18.546573 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:18.548063 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:18.548656 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:18.548768 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:18.549354 (+ 586us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:11:18.550200 (+ 846us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:11:18.550362 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:18.550447 (+ 85us) server_negotiation.cc:299] Negotiation successful
0811 19:11:18.550538 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":772,"thread_start_us":525,"threads_started":1}
I20260811 19:11:18.572640 7711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:18.565264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:52693 (local address 127.3.61.129:46395)
0811 19:11:18.565537 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:18.565542 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:18.566414 (+ 872us) server_negotiation.cc:407] Connection header received
0811 19:11:18.566643 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:18.566647 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:18.566727 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:18.566812 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:11:18.567666 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:18.568657 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:18.570388 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:18.570847 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:18.571009 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:18.571664 (+ 655us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:11:18.571853 (+ 189us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:11:18.571977 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:18.572244 (+ 267us) server_negotiation.cc:299] Negotiation successful
0811 19:11:18.572325 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260811 19:11:18.573053 7738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:18.565246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:46395 (local address 127.3.61.131:52693)
0811 19:11:18.566253 (+ 1007us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:18.566270 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:18.566467 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:18.566883 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:18.566887 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:18.566898 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 19:11:18.567480 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:18.567492 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:18.568815 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:18.568819 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:18.570228 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:18.570243 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:18.571087 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:18.571093 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:18.571243 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:18.571314 (+ 71us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 19:11:18.572025 (+ 711us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 19:11:18.572042 (+ 17us) client_negotiation.cc:769] Sending connection context
0811 19:11:18.572129 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 19:11:18.572255 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":840,"thread_start_us":541,"threads_started":1}
I20260811 19:11:18.585427 7114 raft_consensus.cc:1261] T 6c1befe1fbee42a1b98534838bfafd0e P 5fa054465f054cdcaef05668dd6c801c [term 1 FOLLOWER]: Refusing update from remote peer b60c32f9098646edaf34b986f782a645: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:11:18.585707 6830 raft_consensus.cc:1261] T 6c1befe1fbee42a1b98534838bfafd0e P 5943e4a12ecb475db4d131b5047b8f35 [term 1 FOLLOWER]: Refusing update from remote peer b60c32f9098646edaf34b986f782a645: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:11:18.586362 7730 consensus_queue.cc:1035] T 6c1befe1fbee42a1b98534838bfafd0e P b60c32f9098646edaf34b986f782a645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fa054465f054cdcaef05668dd6c801c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:11:18.586807 7705 consensus_queue.cc:1035] T 6c1befe1fbee42a1b98534838bfafd0e P b60c32f9098646edaf34b986f782a645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5943e4a12ecb475db4d131b5047b8f35" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:11:18.624866 7733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:18.616957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:58733 (local address 127.3.61.131:39303)
0811 19:11:18.617121 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:18.617126 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:18.617595 (+ 469us) server_negotiation.cc:407] Connection header received
0811 19:11:18.617732 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:18.617738 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:18.617828 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:18.617919 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:11:18.618710 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:18.619796 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:18.622223 (+ 2427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:18.623025 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:18.623144 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:18.623904 (+ 760us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:11:18.624061 (+ 157us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:11:18.624170 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:18.624377 (+ 207us) server_negotiation.cc:299] Negotiation successful
0811 19:11:18.624468 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 19:11:18.625087 7748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:18.616362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.131:39303 (local address 127.3.61.129:58733)
0811 19:11:18.617358 (+ 996us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:18.617378 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:18.617526 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:18.617961 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:18.617966 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:18.617978 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 19:11:18.618447 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:18.618458 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:18.620129 (+ 1671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:18.620137 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:18.621979 (+ 1842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:18.622007 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:18.623335 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:18.623342 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:18.623606 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:18.623668 (+ 62us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 19:11:18.624208 (+ 540us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 19:11:18.624222 (+ 14us) client_negotiation.cc:769] Sending connection context
0811 19:11:18.624319 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 19:11:18.624514 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":739,"thread_start_us":534,"threads_started":1}
W20260811 19:11:18.626581 7645 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 19:11:20.268201 6530 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33556:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:11:20.268493 6530 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33556:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:11:20.274251 6530 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 404597c6971d4297830d78a22a801cac: Sending DeleteTablet for 3 replicas of tablet 6c1befe1fbee42a1b98534838bfafd0e
I20260811 19:11:20.275647 7338 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c1befe1fbee42a1b98534838bfafd0e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:11:20 UTC) from {username='kudu', principal='kudu/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:58612
I20260811 19:11:20.276175 6791 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c1befe1fbee42a1b98534838bfafd0e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:11:20 UTC) from {username='kudu', principal='kudu/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:39448
I20260811 19:11:20.276589 7803 tablet_replica.cc:330] T 6c1befe1fbee42a1b98534838bfafd0e P b60c32f9098646edaf34b986f782a645: stopping tablet replica
I20260811 19:11:20.276705 7068 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c1befe1fbee42a1b98534838bfafd0e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:11:20 UTC) from {username='kudu', principal='kudu/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:35282
I20260811 19:11:20.276968 7803 raft_consensus.cc:2229] T 6c1befe1fbee42a1b98534838bfafd0e P b60c32f9098646edaf34b986f782a645 [term 1 LEADER]: Raft consensus shutting down.
I20260811 19:11:20.277439 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 6657
I20260811 19:11:20.277878 7803 raft_consensus.cc:2258] T 6c1befe1fbee42a1b98534838bfafd0e P b60c32f9098646edaf34b986f782a645 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:11:20.279645 7803 ts_tablet_manager.cc:1905] T 6c1befe1fbee42a1b98534838bfafd0e P b60c32f9098646edaf34b986f782a645: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:11:20.288691 7803 ts_tablet_manager.cc:1918] T 6c1befe1fbee42a1b98534838bfafd0e P b60c32f9098646edaf34b986f782a645: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 19:11:20.289034 7803 log.cc:1196] T 6c1befe1fbee42a1b98534838bfafd0e P b60c32f9098646edaf34b986f782a645: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-2/wal/wals/6c1befe1fbee42a1b98534838bfafd0e
I20260811 19:11:20.289907 7803 ts_tablet_manager.cc:1939] T 6c1befe1fbee42a1b98534838bfafd0e P b60c32f9098646edaf34b986f782a645: Deleting consensus metadata
I20260811 19:11:20.291360 6504 catalog_manager.cc:4841] TS b60c32f9098646edaf34b986f782a645 (127.3.61.131:39303): tablet 6c1befe1fbee42a1b98534838bfafd0e (table test-table [id=cfcde4dcded84509804717f30aacb4ca]) successfully deleted
I20260811 19:11:20.292634 7805 tablet_replica.cc:330] T 6c1befe1fbee42a1b98534838bfafd0e P 5fa054465f054cdcaef05668dd6c801c: stopping tablet replica
I20260811 19:11:20.292853 7805 raft_consensus.cc:2229] T 6c1befe1fbee42a1b98534838bfafd0e P 5fa054465f054cdcaef05668dd6c801c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:11:20.293138 7805 raft_consensus.cc:2258] T 6c1befe1fbee42a1b98534838bfafd0e P 5fa054465f054cdcaef05668dd6c801c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:11:20.294145 7805 ts_tablet_manager.cc:1905] T 6c1befe1fbee42a1b98534838bfafd0e P 5fa054465f054cdcaef05668dd6c801c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:11:20.300597 7805 ts_tablet_manager.cc:1918] T 6c1befe1fbee42a1b98534838bfafd0e P 5fa054465f054cdcaef05668dd6c801c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 19:11:20.300787 7805 log.cc:1196] T 6c1befe1fbee42a1b98534838bfafd0e P 5fa054465f054cdcaef05668dd6c801c: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-1/wal/wals/6c1befe1fbee42a1b98534838bfafd0e
I20260811 19:11:20.301422 7805 ts_tablet_manager.cc:1939] T 6c1befe1fbee42a1b98534838bfafd0e P 5fa054465f054cdcaef05668dd6c801c: Deleting consensus metadata
I20260811 19:11:20.302909 6503 catalog_manager.cc:4841] TS 5fa054465f054cdcaef05668dd6c801c (127.3.61.130:38405): tablet 6c1befe1fbee42a1b98534838bfafd0e (table test-table [id=cfcde4dcded84509804717f30aacb4ca]) successfully deleted
I20260811 19:11:20.304724 7804 tablet_replica.cc:330] T 6c1befe1fbee42a1b98534838bfafd0e P 5943e4a12ecb475db4d131b5047b8f35: stopping tablet replica
I20260811 19:11:20.304970 7804 raft_consensus.cc:2229] T 6c1befe1fbee42a1b98534838bfafd0e P 5943e4a12ecb475db4d131b5047b8f35 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:11:20.305330 7804 raft_consensus.cc:2258] T 6c1befe1fbee42a1b98534838bfafd0e P 5943e4a12ecb475db4d131b5047b8f35 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:11:20.306378 7804 ts_tablet_manager.cc:1905] T 6c1befe1fbee42a1b98534838bfafd0e P 5943e4a12ecb475db4d131b5047b8f35: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:11:20.313310 7804 ts_tablet_manager.cc:1918] T 6c1befe1fbee42a1b98534838bfafd0e P 5943e4a12ecb475db4d131b5047b8f35: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 19:11:20.313495 7804 log.cc:1196] T 6c1befe1fbee42a1b98534838bfafd0e P 5943e4a12ecb475db4d131b5047b8f35: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786475454804547-3318-0/minicluster-data/ts-0/wal/wals/6c1befe1fbee42a1b98534838bfafd0e
I20260811 19:11:20.314038 7804 ts_tablet_manager.cc:1939] T 6c1befe1fbee42a1b98534838bfafd0e P 5943e4a12ecb475db4d131b5047b8f35: Deleting consensus metadata
I20260811 19:11:20.316198 6503 catalog_manager.cc:4841] TS 5943e4a12ecb475db4d131b5047b8f35 (127.3.61.129:46395): tablet 6c1befe1fbee42a1b98534838bfafd0e (table test-table [id=cfcde4dcded84509804717f30aacb4ca]) successfully deleted
Aug 11 19:11:20 dist-test-slave-t7g5 krb5kdc[6453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475478, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.3.61.129@KRBTEST.COM
I20260811 19:11:20.326277 7807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:20.280394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39458 (local address 127.3.61.129:46395)
0811 19:11:20.304675 (+ 24281us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:20.304684 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:20.304718 (+ 34us) server_negotiation.cc:407] Connection header received
0811 19:11:20.304780 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:20.304785 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:20.304863 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:20.304984 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:20.306988 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:20.308381 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:20.310318 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:20.311087 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:20.311207 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:20.318210 (+ 7003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:20.318234 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:20.318256 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:20.318295 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:20.322910 (+ 4615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:20.323839 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:20.323851 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:20.323853 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:20.324069 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:20.324552 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:20.324557 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:20.324564 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:20.325024 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:20.325287 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:20.325644 (+ 357us) server_negotiation.cc:299] Negotiation successful
0811 19:11:20.325927 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24116,"thread_start_us":23290,"threads_started":1}
W20260811 19:11:20.327188 6912 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39458
W20260811 19:11:20.327963 3318 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 6657: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 19:11:20.328225 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 6657
I20260811 19:11:20.376950 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 6935
Aug 11 19:11:20 dist-test-slave-t7g5 krb5kdc[6453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475478, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.3.61.130@KRBTEST.COM
I20260811 19:11:20.396752 7808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:20.377935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35312 (local address 127.3.61.130:38405)
0811 19:11:20.378873 (+ 938us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:20.378878 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:20.378901 (+ 23us) server_negotiation.cc:407] Connection header received
0811 19:11:20.378981 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:20.378986 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:20.379088 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:20.379233 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:20.381097 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:20.382275 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:20.383874 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:20.384482 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:20.384604 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:20.389192 (+ 4588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:20.389209 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:20.389227 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:20.389274 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:20.392648 (+ 3374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:20.393544 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:20.393553 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:20.393555 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:20.393787 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:20.394378 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:20.394386 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:20.394397 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:20.395131 (+ 734us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:20.395368 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:20.395946 (+ 578us) server_negotiation.cc:299] Negotiation successful
0811 19:11:20.396267 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":747,"thread_start_us":432,"threads_started":1}
W20260811 19:11:20.397558 7185 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35312
W20260811 19:11:20.398327 3318 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 6935: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 19:11:20.398569 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 6935
I20260811 19:11:20.430677 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 7201
Aug 11 19:11:20 dist-test-slave-t7g5 krb5kdc[6453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475478, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.3.61.131@KRBTEST.COM
I20260811 19:11:20.452803 7812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:20.431454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58632 (local address 127.3.61.131:39303)
0811 19:11:20.431976 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:20.431981 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:20.431997 (+ 16us) server_negotiation.cc:407] Connection header received
0811 19:11:20.432056 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:20.432059 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:20.432143 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:20.432232 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:20.433825 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:20.435005 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:20.436748 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:20.438282 (+ 1534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:20.439382 (+ 1100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:20.444303 (+ 4921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:20.444331 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:20.444360 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:20.444419 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:20.448862 (+ 4443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:20.449891 (+ 1029us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:20.449901 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:20.449905 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:20.450109 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:20.450714 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:20.450720 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:20.450723 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:20.451259 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:20.451613 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:20.452137 (+ 524us) server_negotiation.cc:299] Negotiation successful
0811 19:11:20.452397 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":268,"threads_started":1}
W20260811 19:11:20.453934 7476 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58632
W20260811 19:11:20.454691 3318 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 7201: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 19:11:20.454895 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 7201
I20260811 19:11:20.487771 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 6475
I20260811 19:11:20.489146 6612 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:11:20.620396 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 6475
2026-08-11T19:11:20Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (9163 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:11:20 dist-test-slave-t7g5 krb5kdc[7819](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:11:20 dist-test-slave-t7g5 krb5kdc[7819](info): set up 2 sockets
Aug 11 19:11:20 dist-test-slave-t7g5 krb5kdc[7819](info): commencing operation
krb5kdc: starting...
W20260811 19:11:21.997634 3318 mini_kdc.cc:121] Time spent starting KDC: real 1.322s user 0.005s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260811 19:11:22.127681 3318 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.130s user 0.004s sys 0.008s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260811 19:11:22.278934 3318 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.151s user 0.001s sys 0.009s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260811 19:11:22.409350 3318 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.130s user 0.000s sys 0.009s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/krb5kdc/test-user.keytab.
W20260811 19:11:22.676453 3318 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.267s user 0.004s sys 0.007s
Aug 11 19:11:22 dist-test-slave-t7g5 krb5kdc[7819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475482, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:11:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:11:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.61.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab.
Entry for principal kudu/127.3.61.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab.
I20260811 19:11:22.803779 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:33739
--webserver_interface=127.3.61.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:34901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.61.190:33739
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:11:23.177707 7845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:11:23.178048 7845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:11:23.178166 7845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:11:23.187378 7845 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:11:23.187546 7845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:11:23.187600 7845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:11:23.187688 7845 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:11:23.187752 7845 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:11:23.199921 7845 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:34901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.61.190:33739
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:33739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.61.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:11:23.201542 7845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:11:23.203259 7845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:11:23.217950 7854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:23.217880 7857 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:23.218003 7855 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:11:23.219185 7845 server_base.cc:1034] running on GCE node
I20260811 19:11:23.220168 7845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:11:23.221882 7845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:11:23.223131 7845 hybrid_clock.cc:648] HybridClock initialized: now 1786475483223073 us; error 37 us; skew 500 ppm
I20260811 19:11:23.223541 7845 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:11:23 dist-test-slave-t7g5 krb5kdc[7819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475483, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:11:23.230290 7845 init.cc:377] Logged in from keytab as kudu/127.3.61.190@KRBTEST.COM (short username kudu)
I20260811 19:11:23.232067 7845 webserver.cc:460] Webserver started at http://127.3.61.190:41253/ using document root <none> and password file <none>
I20260811 19:11:23.232847 7845 fs_manager.cc:359] Metadata directory not provided
I20260811 19:11:23.232967 7845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:11:23.233294 7845 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:11:23.236971 7845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/master-0/data/instance:
uuid: "ca2952ae4d16482b800f7629ac783d38"
format_stamp: "Formatted at 2026-08-11 19:11:23 on dist-test-slave-t7g5"
server_key: "c6610c5db095ab921fb2ebc0a57cc2ea"
server_key_iv: "840661b10caf5ac1f9a1ab3996fd2148"
server_key_version: "encryptionkey@0"
I20260811 19:11:23.237952 7845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/master-0/wal/instance:
uuid: "ca2952ae4d16482b800f7629ac783d38"
format_stamp: "Formatted at 2026-08-11 19:11:23 on dist-test-slave-t7g5"
server_key: "c6610c5db095ab921fb2ebc0a57cc2ea"
server_key_iv: "840661b10caf5ac1f9a1ab3996fd2148"
server_key_version: "encryptionkey@0"
I20260811 19:11:23.243762 7845 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 19:11:23.248507 7866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:23.250164 7845 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:11:23.250365 7845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/master-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/master-0/wal
uuid: "ca2952ae4d16482b800f7629ac783d38"
format_stamp: "Formatted at 2026-08-11 19:11:23 on dist-test-slave-t7g5"
server_key: "c6610c5db095ab921fb2ebc0a57cc2ea"
server_key_iv: "840661b10caf5ac1f9a1ab3996fd2148"
server_key_version: "encryptionkey@0"
I20260811 19:11:23.250583 7845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:11:23.269227 7845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:11:23.274979 7845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:11:23.275326 7845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:11:23.298341 7845 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.190:33739
I20260811 19:11:23.298437 7959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.190:33739 every 8 connection(s)
I20260811 19:11:23.300001 7845 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
I20260811 19:11:23.304037 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 7845
I20260811 19:11:23.304317 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/master-0/wal/instance
I20260811 19:11:23.304351 7960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:11:23.305200 3318 external_mini_cluster.cc:1428] Setting key ec4b26779abf81b83598c1ea8f56e8c0
I20260811 19:11:23.319514 7960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca2952ae4d16482b800f7629ac783d38: Bootstrap starting.
Aug 11 19:11:23 dist-test-slave-t7g5 krb5kdc[7819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475482, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:11:23.323323 7960 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca2952ae4d16482b800f7629ac783d38: Neither blocks nor log segments found. Creating new log.
I20260811 19:11:23.324748 7960 log.cc:824] T 00000000000000000000000000000000 P ca2952ae4d16482b800f7629ac783d38: Log is configured to *not* fsync() on all Append() calls
I20260811 19:11:23.329154 7960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca2952ae4d16482b800f7629ac783d38: No bootstrap required, opened a new log
I20260811 19:11:23.331153 7968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:23.307714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56940 (local address 127.3.61.190:33739)
0811 19:11:23.308675 (+ 961us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:23.308687 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:23.308732 (+ 45us) server_negotiation.cc:407] Connection header received
0811 19:11:23.310440 (+ 1708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:23.310465 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:23.310927 (+ 462us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:23.311345 (+ 418us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:23.312958 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:23.314494 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:23.315935 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:23.316755 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:23.316994 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:23.321808 (+ 4814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:23.321820 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:23.321853 (+ 33us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:23.321911 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:23.325682 (+ 3771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:23.326628 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:23.326637 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:23.326646 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:23.326903 (+ 257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:23.327409 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:23.327415 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:23.327418 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:23.328165 (+ 747us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:23.328499 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:23.329168 (+ 669us) server_negotiation.cc:299] Negotiation successful
0811 19:11:23.329716 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":714,"thread_start_us":437,"threads_started":1}
I20260811 19:11:23.339857 7960 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca2952ae4d16482b800f7629ac783d38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca2952ae4d16482b800f7629ac783d38" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33739 } }
I20260811 19:11:23.340613 7960 raft_consensus.cc:374] T 00000000000000000000000000000000 P ca2952ae4d16482b800f7629ac783d38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:11:23.340754 7960 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca2952ae4d16482b800f7629ac783d38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca2952ae4d16482b800f7629ac783d38, State: Initialized, Role: FOLLOWER
I20260811 19:11:23.341596 7960 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca2952ae4d16482b800f7629ac783d38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca2952ae4d16482b800f7629ac783d38" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33739 } }
I20260811 19:11:23.341979 7960 raft_consensus.cc:388] T 00000000000000000000000000000000 P ca2952ae4d16482b800f7629ac783d38 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:11:23.342134 7960 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca2952ae4d16482b800f7629ac783d38 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:11:23.342335 7960 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca2952ae4d16482b800f7629ac783d38 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:11:23.345080 7960 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca2952ae4d16482b800f7629ac783d38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca2952ae4d16482b800f7629ac783d38" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33739 } }
I20260811 19:11:23.345999 7960 leader_election.cc:302] T 00000000000000000000000000000000 P ca2952ae4d16482b800f7629ac783d38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ca2952ae4d16482b800f7629ac783d38; no voters:
I20260811 19:11:23.346957 7960 leader_election.cc:288] T 00000000000000000000000000000000 P ca2952ae4d16482b800f7629ac783d38 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:11:23.347038 7973 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca2952ae4d16482b800f7629ac783d38 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:11:23.347783 7973 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca2952ae4d16482b800f7629ac783d38 [term 1 LEADER]: Becoming Leader. State: Replica: ca2952ae4d16482b800f7629ac783d38, State: Running, Role: LEADER
I20260811 19:11:23.348438 7973 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca2952ae4d16482b800f7629ac783d38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca2952ae4d16482b800f7629ac783d38" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33739 } }
I20260811 19:11:23.349349 7960 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca2952ae4d16482b800f7629ac783d38 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:11:23.352186 7976 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca2952ae4d16482b800f7629ac783d38 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ca2952ae4d16482b800f7629ac783d38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca2952ae4d16482b800f7629ac783d38" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33739 } } }
I20260811 19:11:23.352617 7976 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca2952ae4d16482b800f7629ac783d38 [sys.catalog]: This master's current role is: LEADER
I20260811 19:11:23.353147 7974 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca2952ae4d16482b800f7629ac783d38 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca2952ae4d16482b800f7629ac783d38. Latest consensus state: current_term: 1 leader_uuid: "ca2952ae4d16482b800f7629ac783d38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca2952ae4d16482b800f7629ac783d38" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33739 } } }
I20260811 19:11:23.353449 7974 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca2952ae4d16482b800f7629ac783d38 [sys.catalog]: This master's current role is: LEADER
I20260811 19:11:23.356290 7992 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:11:23.361820 7992 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:11:23.372561 7992 catalog_manager.cc:1347] Generated new cluster ID: 18f70cf64d714547bbbfbe5cd7210ed4
I20260811 19:11:23.372714 7992 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:11:23.385687 7992 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:11:23.387100 7992 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:11:23.409896 7992 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ca2952ae4d16482b800f7629ac783d38: Generated new TSK 0
I20260811 19:11:23.411151 7992 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.61.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab.
Entry for principal kudu/127.3.61.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab.
I20260811 19:11:23.494572 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.129:0
--local_ip_for_outbound_sockets=127.3.61.129
--webserver_interface=127.3.61.129
--webserver_port=0
--tserver_master_addrs=127.3.61.190:33739
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab
--principal=kudu/127.3.61.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:34901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:11:23.892899 8015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:11:23.893232 8015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:11:23.893339 8015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:11:23.901131 8015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:11:23.901295 8015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:11:23.901424 8015 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.129
I20260811 19:11:23.912133 8015 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:34901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab
--principal=kudu/127.3.61.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.61.129
--webserver_port=0
--tserver_master_addrs=127.3.61.190:33739
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.129
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:11:23.913604 8015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:11:23.915024 8015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:11:23.929180 8020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:23.929342 8024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:23.929548 8021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:11:23.930131 8015 server_base.cc:1034] running on GCE node
I20260811 19:11:23.930831 8015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:11:23.931936 8015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:11:23.933167 8015 hybrid_clock.cc:648] HybridClock initialized: now 1786475483933115 us; error 47 us; skew 500 ppm
I20260811 19:11:23.933521 8015 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:11:23 dist-test-slave-t7g5 krb5kdc[7819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475483, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:11:23.937003 8015 init.cc:377] Logged in from keytab as kudu/127.3.61.129@KRBTEST.COM (short username kudu)
I20260811 19:11:23.938747 8015 webserver.cc:460] Webserver started at http://127.3.61.129:37233/ using document root <none> and password file <none>
I20260811 19:11:23.939543 8015 fs_manager.cc:359] Metadata directory not provided
I20260811 19:11:23.939683 8015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:11:23.940034 8015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:11:23.943413 8015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-0/data/instance:
uuid: "b716569f61db4abe90d7c5169c5628ed"
format_stamp: "Formatted at 2026-08-11 19:11:23 on dist-test-slave-t7g5"
server_key: "d8e12bc75d497fc319b7bf325b6b9c4f"
server_key_iv: "006f1607e497c0af83322fca7dc0b915"
server_key_version: "encryptionkey@0"
I20260811 19:11:23.944345 8015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-0/wal/instance:
uuid: "b716569f61db4abe90d7c5169c5628ed"
format_stamp: "Formatted at 2026-08-11 19:11:23 on dist-test-slave-t7g5"
server_key: "d8e12bc75d497fc319b7bf325b6b9c4f"
server_key_iv: "006f1607e497c0af83322fca7dc0b915"
server_key_version: "encryptionkey@0"
I20260811 19:11:23.951189 8015 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260811 19:11:23.955627 8039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:23.957386 8015 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 19:11:23.957594 8015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-0/wal
uuid: "b716569f61db4abe90d7c5169c5628ed"
format_stamp: "Formatted at 2026-08-11 19:11:23 on dist-test-slave-t7g5"
server_key: "d8e12bc75d497fc319b7bf325b6b9c4f"
server_key_iv: "006f1607e497c0af83322fca7dc0b915"
server_key_version: "encryptionkey@0"
I20260811 19:11:23.957904 8015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:11:23.985745 8015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:11:23.990959 8015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:11:23.991273 8015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:11:23.992352 8015 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:11:23.993839 8015 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:11:23.993923 8015 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:23.994058 8015 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:11:23.994114 8015 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:24.032464 8015 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.129:38179
I20260811 19:11:24.032521 8270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.129:38179 every 8 connection(s)
I20260811 19:11:24.033865 8015 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
I20260811 19:11:24.039826 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 8015
I20260811 19:11:24.040059 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-0/wal/instance
I20260811 19:11:24.040693 3318 external_mini_cluster.cc:1428] Setting key f2cb01ed776355e9339d95187141b665
Aug 11 19:11:24 dist-test-slave-t7g5 krb5kdc[7819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475483, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.129@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:11:24.059775 8274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:24.036687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:36977 (local address 127.3.61.190:33739)
0811 19:11:24.037178 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:24.037190 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:24.038286 (+ 1096us) server_negotiation.cc:407] Connection header received
0811 19:11:24.040302 (+ 2016us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:24.040308 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:24.040403 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:24.040503 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:24.042694 (+ 2191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:24.043800 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:24.045969 (+ 2169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:24.046641 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:24.046790 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:24.052403 (+ 5613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:24.052411 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:24.052415 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:24.052474 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:24.055333 (+ 2859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:24.056902 (+ 1569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:24.056906 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:24.056909 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:24.057063 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:24.057747 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:24.057751 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:24.057753 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:24.058098 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:24.058228 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:24.059248 (+ 1020us) server_negotiation.cc:299] Negotiation successful
0811 19:11:24.059460 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":268,"threads_started":1}
I20260811 19:11:24.061152 8278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:24.037228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33739 (local address 127.3.61.129:36977)
0811 19:11:24.038101 (+ 873us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:24.038153 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:24.039974 (+ 1821us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:24.040779 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:24.040791 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:24.041478 (+ 687us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:11:24.042451 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:24.042481 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:24.044143 (+ 1662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:24.044151 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:24.045777 (+ 1626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:24.045806 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:24.047052 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:24.047061 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:24.047409 (+ 348us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:24.048716 (+ 1307us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:11:24.048753 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:11:24.052084 (+ 3331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:11:24.055577 (+ 3493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:24.055601 (+ 24us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:24.055627 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:24.056705 (+ 1078us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:24.057200 (+ 495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:24.057206 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:24.057210 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:24.057551 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:24.058250 (+ 699us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:11:24.058259 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:11:24.058726 (+ 467us) client_negotiation.cc:769] Sending connection context
0811 19:11:24.059058 (+ 332us) client_negotiation.cc:240] Negotiation successful
0811 19:11:24.059712 (+ 654us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":379,"threads_started":1}
I20260811 19:11:24.063182 8271 heartbeater.cc:347] Connected to a master server at 127.3.61.190:33739
I20260811 19:11:24.063550 8271 heartbeater.cc:464] Registering TS with master...
I20260811 19:11:24.064440 8271 heartbeater.cc:511] Master 127.3.61.190:33739 requested a full tablet report, sending...
I20260811 19:11:24.066833 7897 ts_manager.cc:194] Registered new tserver with Master: b716569f61db4abe90d7c5169c5628ed (127.3.61.129:38179)
I20260811 19:11:24.068395 7897 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.129@KRBTEST.COM'} at 127.3.61.129:36977
WARNING: no policy specified for kudu/127.3.61.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab.
Entry for principal kudu/127.3.61.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab.
I20260811 19:11:24.101996 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.130:0
--local_ip_for_outbound_sockets=127.3.61.130
--webserver_interface=127.3.61.130
--webserver_port=0
--tserver_master_addrs=127.3.61.190:33739
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab
--principal=kudu/127.3.61.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:34901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:11:24.461359 8289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:11:24.461719 8289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:11:24.461871 8289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:11:24.469961 8289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:11:24.470086 8289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:11:24.470203 8289 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.130
I20260811 19:11:24.480400 8289 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:34901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab
--principal=kudu/127.3.61.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.61.130
--webserver_port=0
--tserver_master_addrs=127.3.61.190:33739
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.130
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:11:24.481784 8289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:11:24.483284 8289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:11:24.497326 8301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:24.497448 8299 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:24.497476 8306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:11:24.498569 8289 server_base.cc:1034] running on GCE node
I20260811 19:11:24.499296 8289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:11:24.500367 8289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:11:24.501607 8289 hybrid_clock.cc:648] HybridClock initialized: now 1786475484501531 us; error 72 us; skew 500 ppm
I20260811 19:11:24.501984 8289 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:11:24 dist-test-slave-t7g5 krb5kdc[7819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475484, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:11:24.506213 8289 init.cc:377] Logged in from keytab as kudu/127.3.61.130@KRBTEST.COM (short username kudu)
I20260811 19:11:24.508723 8289 webserver.cc:460] Webserver started at http://127.3.61.130:41051/ using document root <none> and password file <none>
I20260811 19:11:24.509516 8289 fs_manager.cc:359] Metadata directory not provided
I20260811 19:11:24.509646 8289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:11:24.510033 8289 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:11:24.513269 8289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-1/data/instance:
uuid: "c7a763a1419840b989f4fe1dc9974747"
format_stamp: "Formatted at 2026-08-11 19:11:24 on dist-test-slave-t7g5"
server_key: "271d2e43e465648c32b32a3404856fff"
server_key_iv: "84b90cb183c48cf23abae70535da5ae8"
server_key_version: "encryptionkey@0"
I20260811 19:11:24.514163 8289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-1/wal/instance:
uuid: "c7a763a1419840b989f4fe1dc9974747"
format_stamp: "Formatted at 2026-08-11 19:11:24 on dist-test-slave-t7g5"
server_key: "271d2e43e465648c32b32a3404856fff"
server_key_iv: "84b90cb183c48cf23abae70535da5ae8"
server_key_version: "encryptionkey@0"
I20260811 19:11:24.520470 8289 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260811 19:11:24.524294 8315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:24.525827 8289 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:11:24.525987 8289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-1/wal
uuid: "c7a763a1419840b989f4fe1dc9974747"
format_stamp: "Formatted at 2026-08-11 19:11:24 on dist-test-slave-t7g5"
server_key: "271d2e43e465648c32b32a3404856fff"
server_key_iv: "84b90cb183c48cf23abae70535da5ae8"
server_key_version: "encryptionkey@0"
I20260811 19:11:24.526194 8289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:11:24.560011 8289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:11:24.565021 8289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:11:24.565336 8289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:11:24.566506 8289 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:11:24.568029 8289 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:11:24.568120 8289 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:24.568231 8289 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:11:24.568282 8289 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:24.606254 8289 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.130:41731
I20260811 19:11:24.606297 8571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.130:41731 every 8 connection(s)
I20260811 19:11:24.607710 8289 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
I20260811 19:11:24.611341 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 8289
I20260811 19:11:24.611557 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-1/wal/instance
I20260811 19:11:24.612018 3318 external_mini_cluster.cc:1428] Setting key 0d370469ce4f4ea61899001e2eaf45d5
Aug 11 19:11:24 dist-test-slave-t7g5 krb5kdc[7819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475484, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.130@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:11:24.633796 8578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:24.610610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:55583 (local address 127.3.61.190:33739)
0811 19:11:24.611226 (+ 616us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:24.611232 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:24.612183 (+ 951us) server_negotiation.cc:407] Connection header received
0811 19:11:24.614261 (+ 2078us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:24.614267 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:24.614372 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:24.614490 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:24.617155 (+ 2665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:24.618261 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:24.620015 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:24.620609 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:24.620743 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:24.625609 (+ 4866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:24.625615 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:24.625619 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:24.625687 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:24.628262 (+ 2575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:24.630224 (+ 1962us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:24.630231 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:24.630234 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:24.630478 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:24.631362 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:24.631368 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:24.631370 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:24.631939 (+ 569us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:24.632135 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:24.633205 (+ 1070us) server_negotiation.cc:299] Negotiation successful
0811 19:11:24.633472 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":304,"threads_started":1}
I20260811 19:11:24.635187 8580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:24.611098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33739 (local address 127.3.61.130:55583)
0811 19:11:24.611942 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:24.611990 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:24.613991 (+ 2001us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:24.614743 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:24.614751 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:24.615814 (+ 1063us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:11:24.616815 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:24.616845 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:24.618488 (+ 1643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:24.618492 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:24.619834 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:24.619849 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:24.620978 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:24.620989 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:24.621226 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:24.622053 (+ 827us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:11:24.622085 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:11:24.625304 (+ 3219us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:11:24.628571 (+ 3267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:24.628588 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:24.628617 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:24.629971 (+ 1354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:24.630792 (+ 821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:24.630797 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:24.630800 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:24.631158 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:24.632119 (+ 961us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:11:24.632128 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:11:24.632640 (+ 512us) client_negotiation.cc:769] Sending connection context
0811 19:11:24.632982 (+ 342us) client_negotiation.cc:240] Negotiation successful
0811 19:11:24.633585 (+ 603us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":349,"threads_started":1}
I20260811 19:11:24.637002 8574 heartbeater.cc:347] Connected to a master server at 127.3.61.190:33739
I20260811 19:11:24.637346 8574 heartbeater.cc:464] Registering TS with master...
I20260811 19:11:24.638170 8574 heartbeater.cc:511] Master 127.3.61.190:33739 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.3.61.131@KRBTEST.COM; defaulting to no policy
I20260811 19:11:24.639878 7897 ts_manager.cc:194] Registered new tserver with Master: c7a763a1419840b989f4fe1dc9974747 (127.3.61.130:41731)
I20260811 19:11:24.640899 7897 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.130@KRBTEST.COM'} at 127.3.61.130:55583
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab.
Entry for principal kudu/127.3.61.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab.
I20260811 19:11:24.674548 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.131:0
--local_ip_for_outbound_sockets=127.3.61.131
--webserver_interface=127.3.61.131
--webserver_port=0
--tserver_master_addrs=127.3.61.190:33739
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab
--principal=kudu/127.3.61.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:34901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:11:25.026127 8583 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:11:25.026474 8583 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:11:25.026607 8583 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:11:25.034744 8583 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:11:25.034884 8583 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:11:25.035043 8583 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.131
I20260811 19:11:25.045715 8583 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:34901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab
--principal=kudu/127.3.61.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.61.131
--webserver_port=0
--tserver_master_addrs=127.3.61.190:33739
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.131
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:11:25.047119 8583 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:11:25.048494 8583 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:11:25.064164 8593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:25.064095 8596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:25.064137 8602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:11:25.065461 8583 server_base.cc:1034] running on GCE node
I20260811 19:11:25.066465 8583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:11:25.067796 8583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:11:25.069059 8583 hybrid_clock.cc:648] HybridClock initialized: now 1786475485069011 us; error 37 us; skew 500 ppm
I20260811 19:11:25.069402 8583 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 19:11:25.071976 8271 heartbeater.cc:503] Master 127.3.61.190:33739 was elected leader, sending a full tablet report...
Aug 11 19:11:25 dist-test-slave-t7g5 krb5kdc[7819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475485, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:11:25.073012 8583 init.cc:377] Logged in from keytab as kudu/127.3.61.131@KRBTEST.COM (short username kudu)
I20260811 19:11:25.075222 8583 webserver.cc:460] Webserver started at http://127.3.61.131:39343/ using document root <none> and password file <none>
I20260811 19:11:25.075961 8583 fs_manager.cc:359] Metadata directory not provided
I20260811 19:11:25.076074 8583 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:11:25.076339 8583 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:11:25.080263 8583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-2/data/instance:
uuid: "477b53dbd79948a68ca235707c62a548"
format_stamp: "Formatted at 2026-08-11 19:11:25 on dist-test-slave-t7g5"
server_key: "6400c007dd4741398c6d11fcfbb93b9a"
server_key_iv: "2d35271bd747aa4cbf70b3a6d9c44f69"
server_key_version: "encryptionkey@0"
I20260811 19:11:25.081099 8583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-2/wal/instance:
uuid: "477b53dbd79948a68ca235707c62a548"
format_stamp: "Formatted at 2026-08-11 19:11:25 on dist-test-slave-t7g5"
server_key: "6400c007dd4741398c6d11fcfbb93b9a"
server_key_iv: "2d35271bd747aa4cbf70b3a6d9c44f69"
server_key_version: "encryptionkey@0"
I20260811 19:11:25.086728 8583 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 19:11:25.090667 8621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:25.092042 8583 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 19:11:25.092211 8583 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-2/wal
uuid: "477b53dbd79948a68ca235707c62a548"
format_stamp: "Formatted at 2026-08-11 19:11:25 on dist-test-slave-t7g5"
server_key: "6400c007dd4741398c6d11fcfbb93b9a"
server_key_iv: "2d35271bd747aa4cbf70b3a6d9c44f69"
server_key_version: "encryptionkey@0"
I20260811 19:11:25.092410 8583 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:11:25.110054 8583 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:11:25.114490 8583 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:11:25.115073 8583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:11:25.116560 8583 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:11:25.118431 8583 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:11:25.118541 8583 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:25.118665 8583 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:11:25.118731 8583 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:25.152721 8583 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.131:40239
I20260811 19:11:25.152763 8832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.131:40239 every 8 connection(s)
I20260811 19:11:25.154081 8583 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
I20260811 19:11:25.161597 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 8583
I20260811 19:11:25.161831 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-2/wal/instance
I20260811 19:11:25.162410 3318 external_mini_cluster.cc:1428] Setting key 4e2aea2df76d6b13a6473bd6d19311b0
Aug 11 19:11:25 dist-test-slave-t7g5 krb5kdc[7819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475485, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.131@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:11:25.177420 8838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:25.157113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:40177 (local address 127.3.61.190:33739)
0811 19:11:25.157567 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:25.157572 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:25.158664 (+ 1092us) server_negotiation.cc:407] Connection header received
0811 19:11:25.160647 (+ 1983us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:25.160652 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:25.160730 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:25.160826 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:25.162921 (+ 2095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:25.164055 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:25.166101 (+ 2046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:25.166531 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:25.166637 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:25.170309 (+ 3672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:25.170315 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:25.170318 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:25.170353 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:25.172961 (+ 2608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:25.173744 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:25.173749 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:25.173752 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:25.173922 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:25.174588 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:25.174592 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:25.174595 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:25.174935 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:25.175087 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:25.176828 (+ 1741us) server_negotiation.cc:299] Negotiation successful
0811 19:11:25.177076 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":226,"threads_started":1}
I20260811 19:11:25.177634 8839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:25.157708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33739 (local address 127.3.61.131:40177)
0811 19:11:25.158490 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:25.158533 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:25.160362 (+ 1829us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:25.161075 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:25.161083 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:25.161906 (+ 823us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:11:25.162696 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:25.162723 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:25.164240 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:25.164246 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:25.165962 (+ 1716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:25.165976 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:25.166790 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:25.166796 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:25.167017 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:25.167687 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:11:25.167710 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:11:25.170068 (+ 2358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:11:25.173106 (+ 3038us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:25.173116 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:25.173133 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:25.173571 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:25.174060 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:25.174063 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:25.174066 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:25.174317 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:25.175138 (+ 821us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:11:25.175159 (+ 21us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:11:25.175546 (+ 387us) client_negotiation.cc:769] Sending connection context
0811 19:11:25.175797 (+ 251us) client_negotiation.cc:240] Negotiation successful
0811 19:11:25.176224 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":306,"threads_started":1}
I20260811 19:11:25.179141 8833 heartbeater.cc:347] Connected to a master server at 127.3.61.190:33739
I20260811 19:11:25.180244 8833 heartbeater.cc:464] Registering TS with master...
I20260811 19:11:25.181054 8833 heartbeater.cc:511] Master 127.3.61.190:33739 requested a full tablet report, sending...
I20260811 19:11:25.182713 7897 ts_manager.cc:194] Registered new tserver with Master: 477b53dbd79948a68ca235707c62a548 (127.3.61.131:40239)
I20260811 19:11:25.183557 7897 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.131@KRBTEST.COM'} at 127.3.61.131:40177
I20260811 19:11:25.192049 3318 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 19:11:25.246414 8838 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:11:25.243801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34478 (local address 127.3.61.190:33739)
0811 19:11:25.244003 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:25.244008 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:25.244388 (+ 380us) server_negotiation.cc:407] Connection header received
0811 19:11:25.244564 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:25.244569 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:25.244640 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:25.244743 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:25.246196 (+ 1453us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34478: BlockingRecv error: recv got EOF from 127.0.0.1:34478 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
I20260811 19:11:25.250497 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 8015
W20260811 19:11:25.253994 3318 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 8015: Not authorized: Client connection negotiation failed: client connection to 127.3.61.129:38179: server requires authentication, but client does not have Kerberos credentials available
I20260811 19:11:25.254194 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 8015
W20260811 19:11:25.254238 8855 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:11:25.251760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35316 (local address 127.3.61.129:38179)
0811 19:11:25.252386 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:25.252393 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:25.252419 (+ 26us) server_negotiation.cc:407] Connection header received
0811 19:11:25.252479 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:25.252485 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:25.252685 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:25.252832 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:25.253841 (+ 1009us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35316: BlockingRecv error: recv got EOF from 127.0.0.1:35316 (error 108)
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":307,"threads_started":1}
I20260811 19:11:25.279358 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 8289
W20260811 19:11:25.282816 3318 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 8289: Not authorized: Client connection negotiation failed: client connection to 127.3.61.130:41731: server requires authentication, but client does not have Kerberos credentials available
I20260811 19:11:25.282995 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 8289
I20260811 19:11:25.311517 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 8583
W20260811 19:11:25.314900 3318 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 8583: Not authorized: Client connection negotiation failed: client connection to 127.3.61.131:40239: server requires authentication, but client does not have Kerberos credentials available
I20260811 19:11:25.315062 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 8583
I20260811 19:11:25.348882 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 7845
I20260811 19:11:25.350250 7951 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:11:25.464601 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 7845
2026-08-11T19:11:25Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (4838 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786475454804547-3318-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:11:25 dist-test-slave-t7g5 krb5kdc[8865](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:11:25 dist-test-slave-t7g5 krb5kdc[8865](info): set up 2 sockets
Aug 11 19:11:25 dist-test-slave-t7g5 krb5kdc[8865](info): commencing operation
krb5kdc: starting...
W20260811 19:11:26.764135 3318 mini_kdc.cc:121] Time spent starting KDC: real 1.250s user 0.005s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Aug 11 19:11:26 dist-test-slave-t7g5 krb5kdc[8865](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475486, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:11:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:11:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.61.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab.
Entry for principal kudu/127.3.61.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab.
I20260811 19:11:27.062965 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786475454804547-3318-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786475454804547-3318-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786475454804547-3318-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:37235
--webserver_interface=127.3.61.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:35263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.61.190:37235
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:11:27.450768 8890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:11:27.451126 8890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:11:27.451249 8890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:11:27.460716 8890 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:11:27.460875 8890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:11:27.460937 8890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:11:27.461027 8890 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:11:27.461091 8890 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:11:27.473798 8890 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:35263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786475454804547-3318-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786475454804547-3318-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.61.190:37235
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:37235
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.61.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786475454804547-3318-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:11:27.475436 8890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:11:27.477631 8890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:11:27.492683 8901 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:27.492771 8898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:27.492717 8903 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:11:27.493891 8890 server_base.cc:1034] running on GCE node
I20260811 19:11:27.494835 8890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:11:27.496510 8890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:11:27.497943 8890 hybrid_clock.cc:648] HybridClock initialized: now 1786475487497828 us; error 84 us; skew 500 ppm
I20260811 19:11:27.498498 8890 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:11:27 dist-test-slave-t7g5 krb5kdc[8865](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475487, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:11:27.502756 8890 init.cc:377] Logged in from keytab as kudu/127.3.61.190@KRBTEST.COM (short username kudu)
I20260811 19:11:27.505110 8890 webserver.cc:460] Webserver started at http://127.3.61.190:33061/ using document root <none> and password file <none>
I20260811 19:11:27.506045 8890 fs_manager.cc:359] Metadata directory not provided
I20260811 19:11:27.506191 8890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:11:27.506528 8890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:11:27.510075 8890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786475454804547-3318-0/minicluster-data/master-0/data/instance:
uuid: "ca2c867120e7451aa2aee0f0c56abf50"
format_stamp: "Formatted at 2026-08-11 19:11:27 on dist-test-slave-t7g5"
server_key: "8cf6c57ed0c2aa3b7d41e28e107e1291"
server_key_iv: "b3a5ba6dad9026eed45c3570840f7c8c"
server_key_version: "encryptionkey@0"
I20260811 19:11:27.510974 8890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786475454804547-3318-0/minicluster-data/master-0/wal/instance:
uuid: "ca2c867120e7451aa2aee0f0c56abf50"
format_stamp: "Formatted at 2026-08-11 19:11:27 on dist-test-slave-t7g5"
server_key: "8cf6c57ed0c2aa3b7d41e28e107e1291"
server_key_iv: "b3a5ba6dad9026eed45c3570840f7c8c"
server_key_version: "encryptionkey@0"
I20260811 19:11:27.517484 8890 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260811 19:11:27.522080 8918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:27.524091 8890 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 19:11:27.524344 8890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786475454804547-3318-0/minicluster-data/master-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786475454804547-3318-0/minicluster-data/master-0/wal
uuid: "ca2c867120e7451aa2aee0f0c56abf50"
format_stamp: "Formatted at 2026-08-11 19:11:27 on dist-test-slave-t7g5"
server_key: "8cf6c57ed0c2aa3b7d41e28e107e1291"
server_key_iv: "b3a5ba6dad9026eed45c3570840f7c8c"
server_key_version: "encryptionkey@0"
I20260811 19:11:27.524580 8890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786475454804547-3318-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786475454804547-3318-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786475454804547-3318-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:11:27.546339 8890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:11:27.551146 8890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:11:27.551442 8890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:11:27.573848 8890 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.190:37235
I20260811 19:11:27.573640 9029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.190:37235 every 8 connection(s)
I20260811 19:11:27.576061 8890 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
I20260811 19:11:27.581698 9030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:11:27.584146 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 8890
I20260811 19:11:27.584376 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786475454804547-3318-0/minicluster-data/master-0/wal/instance
I20260811 19:11:27.585011 3318 external_mini_cluster.cc:1428] Setting key a6dcef54fae88011576bc8a43a5438bb
I20260811 19:11:27.595328 9030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca2c867120e7451aa2aee0f0c56abf50: Bootstrap starting.
I20260811 19:11:27.600812 9030 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca2c867120e7451aa2aee0f0c56abf50: Neither blocks nor log segments found. Creating new log.
Aug 11 19:11:27 dist-test-slave-t7g5 krb5kdc[8865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475486, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:11:27.602406 9030 log.cc:824] T 00000000000000000000000000000000 P ca2c867120e7451aa2aee0f0c56abf50: Log is configured to *not* fsync() on all Append() calls
I20260811 19:11:27.606974 9030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca2c867120e7451aa2aee0f0c56abf50: No bootstrap required, opened a new log
I20260811 19:11:27.613694 9034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:27.587080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59312 (local address 127.3.61.190:37235)
0811 19:11:27.587949 (+ 869us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:27.587962 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:27.588003 (+ 41us) server_negotiation.cc:407] Connection header received
0811 19:11:27.589751 (+ 1748us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:27.589775 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:27.590182 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:27.590536 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:27.592437 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:27.593963 (+ 1526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:27.596302 (+ 2339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:27.597183 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:27.597360 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:27.603855 (+ 6495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:27.603877 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:27.603905 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:27.603967 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:27.608121 (+ 4154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:27.609097 (+ 976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:27.609107 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:27.609121 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:27.609371 (+ 250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:27.609886 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:27.609892 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:27.609896 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:27.610743 (+ 847us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:27.611060 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:27.611592 (+ 532us) server_negotiation.cc:299] Negotiation successful
0811 19:11:27.612102 (+ 510us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":637,"thread_start_us":393,"threads_started":1}
I20260811 19:11:27.618371 9030 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca2c867120e7451aa2aee0f0c56abf50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca2c867120e7451aa2aee0f0c56abf50" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 37235 } }
I20260811 19:11:27.618901 9030 raft_consensus.cc:374] T 00000000000000000000000000000000 P ca2c867120e7451aa2aee0f0c56abf50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:11:27.619024 9030 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca2c867120e7451aa2aee0f0c56abf50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca2c867120e7451aa2aee0f0c56abf50, State: Initialized, Role: FOLLOWER
I20260811 19:11:27.619951 9030 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca2c867120e7451aa2aee0f0c56abf50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca2c867120e7451aa2aee0f0c56abf50" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 37235 } }
I20260811 19:11:27.620236 3318 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:11:27.620390 9030 raft_consensus.cc:388] T 00000000000000000000000000000000 P ca2c867120e7451aa2aee0f0c56abf50 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:11:27.620572 9030 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca2c867120e7451aa2aee0f0c56abf50 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:11:27.620791 9030 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca2c867120e7451aa2aee0f0c56abf50 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:11:27.624564 9030 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca2c867120e7451aa2aee0f0c56abf50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca2c867120e7451aa2aee0f0c56abf50" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 37235 } }
I20260811 19:11:27.625501 9030 leader_election.cc:302] T 00000000000000000000000000000000 P ca2c867120e7451aa2aee0f0c56abf50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ca2c867120e7451aa2aee0f0c56abf50; no voters:
I20260811 19:11:27.626466 9030 leader_election.cc:288] T 00000000000000000000000000000000 P ca2c867120e7451aa2aee0f0c56abf50 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:11:27.626643 9039 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca2c867120e7451aa2aee0f0c56abf50 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:11:27.627415 9039 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca2c867120e7451aa2aee0f0c56abf50 [term 1 LEADER]: Becoming Leader. State: Replica: ca2c867120e7451aa2aee0f0c56abf50, State: Running, Role: LEADER
I20260811 19:11:27.628216 9039 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca2c867120e7451aa2aee0f0c56abf50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca2c867120e7451aa2aee0f0c56abf50" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 37235 } }
I20260811 19:11:27.630343 9030 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca2c867120e7451aa2aee0f0c56abf50 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:11:27.631834 9044 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca2c867120e7451aa2aee0f0c56abf50 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ca2c867120e7451aa2aee0f0c56abf50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca2c867120e7451aa2aee0f0c56abf50" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 37235 } } }
I20260811 19:11:27.632268 9044 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca2c867120e7451aa2aee0f0c56abf50 [sys.catalog]: This master's current role is: LEADER
I20260811 19:11:27.632843 9041 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca2c867120e7451aa2aee0f0c56abf50 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca2c867120e7451aa2aee0f0c56abf50. Latest consensus state: current_term: 1 leader_uuid: "ca2c867120e7451aa2aee0f0c56abf50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca2c867120e7451aa2aee0f0c56abf50" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 37235 } } }
I20260811 19:11:27.633169 9041 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca2c867120e7451aa2aee0f0c56abf50 [sys.catalog]: This master's current role is: LEADER
I20260811 19:11:27.635970 9050 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:11:27.642863 9050 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:11:27.667387 9050 catalog_manager.cc:1347] Generated new cluster ID: 0038263ee7dd4730b6a9088e7af78f95
I20260811 19:11:27.667539 9050 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:11:27.692164 9050 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:11:27.693241 9034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:27.684365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59314 (local address 127.3.61.190:37235)
0811 19:11:27.684627 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:27.684634 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:27.685325 (+ 691us) server_negotiation.cc:407] Connection header received
0811 19:11:27.685513 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:27.685517 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:27.685591 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:27.685708 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:27.687071 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:27.687857 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:27.689193 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:27.689777 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:27.689891 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:27.691539 (+ 1648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:27.691546 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:27.691549 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:11:27.692434 (+ 885us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:27.692514 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:11:27.692540 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:27.692642 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:27.692779 (+ 137us) server_negotiation.cc:299] Negotiation successful
0811 19:11:27.692897 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260811 19:11:27.694187 9050 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:11:27.707479 9050 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ca2c867120e7451aa2aee0f0c56abf50: Generated new TSK 0
I20260811 19:11:27.708658 9050 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:11:27.716674 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 8890
I20260811 19:11:27.718221 9021 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:11:27.817518 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 8890
2026-08-11T19:11:27Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2349 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:11:27 dist-test-slave-t7g5 krb5kdc[9096](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:11:27 dist-test-slave-t7g5 krb5kdc[9096](info): set up 2 sockets
Aug 11 19:11:27 dist-test-slave-t7g5 krb5kdc[9096](info): commencing operation
krb5kdc: starting...
W20260811 19:11:29.117910 3318 mini_kdc.cc:121] Time spent starting KDC: real 1.253s user 0.008s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Aug 11 19:11:29 dist-test-slave-t7g5 krb5kdc[9096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475489, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:11:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:11:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.61.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab.
Entry for principal kudu/127.3.61.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab.
I20260811 19:11:29.401795 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:33103
--webserver_interface=127.3.61.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:39965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.61.190:33103
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:11:29.792390 9119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:11:29.792723 9119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:11:29.792850 9119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:11:29.802567 9119 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:11:29.802755 9119 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:11:29.802811 9119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:11:29.802902 9119 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:11:29.802949 9119 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:11:29.814913 9119 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:39965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.61.190:33103
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:33103
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.61.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:11:29.816452 9119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:11:29.817941 9119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:11:29.837740 9127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:29.838152 9129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:11:29.838379 9119 server_base.cc:1034] running on GCE node
W20260811 19:11:29.838977 9126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:11:29.840440 9119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:11:29.842742 9119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:11:29.844027 9119 hybrid_clock.cc:648] HybridClock initialized: now 1786475489843958 us; error 58 us; skew 500 ppm
I20260811 19:11:29.844431 9119 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:11:29 dist-test-slave-t7g5 krb5kdc[9096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475489, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:11:29.852578 9119 init.cc:377] Logged in from keytab as kudu/127.3.61.190@KRBTEST.COM (short username kudu)
I20260811 19:11:29.854594 9119 webserver.cc:460] Webserver started at http://127.3.61.190:33915/ using document root <none> and password file <none>
I20260811 19:11:29.855692 9119 fs_manager.cc:359] Metadata directory not provided
I20260811 19:11:29.855849 9119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:11:29.856211 9119 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:11:29.859647 9119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/master-0/data/instance:
uuid: "4ecf7bd499f849ad82f8ed9f0911de55"
format_stamp: "Formatted at 2026-08-11 19:11:29 on dist-test-slave-t7g5"
server_key: "50cce345c119ee16a005bdc1ea71d665"
server_key_iv: "90477b4d6af19265ef47305be27591ad"
server_key_version: "encryptionkey@0"
I20260811 19:11:29.860509 9119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/master-0/wal/instance:
uuid: "4ecf7bd499f849ad82f8ed9f0911de55"
format_stamp: "Formatted at 2026-08-11 19:11:29 on dist-test-slave-t7g5"
server_key: "50cce345c119ee16a005bdc1ea71d665"
server_key_iv: "90477b4d6af19265ef47305be27591ad"
server_key_version: "encryptionkey@0"
I20260811 19:11:29.866947 9119 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260811 19:11:29.871105 9140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:29.873010 9119 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.002s
I20260811 19:11:29.873360 9119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/master-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/master-0/wal
uuid: "4ecf7bd499f849ad82f8ed9f0911de55"
format_stamp: "Formatted at 2026-08-11 19:11:29 on dist-test-slave-t7g5"
server_key: "50cce345c119ee16a005bdc1ea71d665"
server_key_iv: "90477b4d6af19265ef47305be27591ad"
server_key_version: "encryptionkey@0"
I20260811 19:11:29.873643 9119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:11:29.892305 9119 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:11:29.897954 9119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:11:29.898290 9119 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:11:29.920454 9119 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.190:33103
I20260811 19:11:29.920621 9262 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.190:33103 every 8 connection(s)
I20260811 19:11:29.922056 9119 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
I20260811 19:11:29.924017 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 9119
I20260811 19:11:29.924258 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/master-0/wal/instance
I20260811 19:11:29.925096 3318 external_mini_cluster.cc:1428] Setting key 7ae6c96feb33c43c8a2f97ebc05bfc4f
I20260811 19:11:29.927608 9263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 19:11:29 dist-test-slave-t7g5 krb5kdc[9096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475489, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:11:29.942651 9263 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ecf7bd499f849ad82f8ed9f0911de55: Bootstrap starting.
I20260811 19:11:29.946825 9263 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4ecf7bd499f849ad82f8ed9f0911de55: Neither blocks nor log segments found. Creating new log.
I20260811 19:11:29.947980 9263 log.cc:824] T 00000000000000000000000000000000 P 4ecf7bd499f849ad82f8ed9f0911de55: Log is configured to *not* fsync() on all Append() calls
I20260811 19:11:29.951730 9269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:29.927258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53268 (local address 127.3.61.190:33103)
0811 19:11:29.928057 (+ 799us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:29.928070 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:29.928111 (+ 41us) server_negotiation.cc:407] Connection header received
0811 19:11:29.929860 (+ 1749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:29.929885 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:29.930290 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:29.930646 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:29.932556 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:29.934407 (+ 1851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:29.936630 (+ 2223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:29.937505 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:29.937732 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:29.943327 (+ 5595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:29.943340 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:29.943364 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:29.943424 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:29.946859 (+ 3435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:29.947728 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:29.947736 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:29.947743 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:29.947940 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:29.948353 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:29.948357 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:29.948359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:29.949163 (+ 804us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:29.949502 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:29.950259 (+ 757us) server_negotiation.cc:299] Negotiation successful
0811 19:11:29.950715 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":341,"threads_started":1}
I20260811 19:11:29.951856 9263 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ecf7bd499f849ad82f8ed9f0911de55: No bootstrap required, opened a new log
I20260811 19:11:29.960172 9263 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ecf7bd499f849ad82f8ed9f0911de55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ecf7bd499f849ad82f8ed9f0911de55" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33103 } }
I20260811 19:11:29.960568 9263 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4ecf7bd499f849ad82f8ed9f0911de55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:11:29.960657 9263 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ecf7bd499f849ad82f8ed9f0911de55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ecf7bd499f849ad82f8ed9f0911de55, State: Initialized, Role: FOLLOWER
I20260811 19:11:29.961284 9263 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ecf7bd499f849ad82f8ed9f0911de55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ecf7bd499f849ad82f8ed9f0911de55" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33103 } }
I20260811 19:11:29.961580 9263 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4ecf7bd499f849ad82f8ed9f0911de55 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:11:29.961751 9263 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ecf7bd499f849ad82f8ed9f0911de55 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:11:29.961930 9263 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ecf7bd499f849ad82f8ed9f0911de55 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:11:29.964136 9263 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ecf7bd499f849ad82f8ed9f0911de55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ecf7bd499f849ad82f8ed9f0911de55" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33103 } }
I20260811 19:11:29.964690 9263 leader_election.cc:302] T 00000000000000000000000000000000 P 4ecf7bd499f849ad82f8ed9f0911de55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4ecf7bd499f849ad82f8ed9f0911de55; no voters:
I20260811 19:11:29.965323 9263 leader_election.cc:288] T 00000000000000000000000000000000 P 4ecf7bd499f849ad82f8ed9f0911de55 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:11:29.965473 9277 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ecf7bd499f849ad82f8ed9f0911de55 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:11:29.966142 9277 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ecf7bd499f849ad82f8ed9f0911de55 [term 1 LEADER]: Becoming Leader. State: Replica: 4ecf7bd499f849ad82f8ed9f0911de55, State: Running, Role: LEADER
I20260811 19:11:29.966677 9277 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ecf7bd499f849ad82f8ed9f0911de55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ecf7bd499f849ad82f8ed9f0911de55" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33103 } }
I20260811 19:11:29.967479 9263 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ecf7bd499f849ad82f8ed9f0911de55 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:11:29.969704 9278 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ecf7bd499f849ad82f8ed9f0911de55 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4ecf7bd499f849ad82f8ed9f0911de55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ecf7bd499f849ad82f8ed9f0911de55" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33103 } } }
I20260811 19:11:29.969841 9284 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ecf7bd499f849ad82f8ed9f0911de55 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ecf7bd499f849ad82f8ed9f0911de55. Latest consensus state: current_term: 1 leader_uuid: "4ecf7bd499f849ad82f8ed9f0911de55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ecf7bd499f849ad82f8ed9f0911de55" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33103 } } }
I20260811 19:11:29.970117 9278 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ecf7bd499f849ad82f8ed9f0911de55 [sys.catalog]: This master's current role is: LEADER
I20260811 19:11:29.970207 9284 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ecf7bd499f849ad82f8ed9f0911de55 [sys.catalog]: This master's current role is: LEADER
I20260811 19:11:29.974022 9289 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:11:29.980512 9289 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:11:29.991268 9289 catalog_manager.cc:1347] Generated new cluster ID: 24f40a3158f84cc1825426e9e3d0cfed
I20260811 19:11:29.991441 9289 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:11:30.024434 9289 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:11:30.025992 9289 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:11:30.039089 9289 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4ecf7bd499f849ad82f8ed9f0911de55: Generated new TSK 0
I20260811 19:11:30.040184 9289 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.61.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab.
Entry for principal kudu/127.3.61.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab.
I20260811 19:11:30.117947 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.129:0
--local_ip_for_outbound_sockets=127.3.61.129
--webserver_interface=127.3.61.129
--webserver_port=0
--tserver_master_addrs=127.3.61.190:33103
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab
--principal=kudu/127.3.61.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:39965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:11:30.503335 9314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:11:30.503669 9314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:11:30.503798 9314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:11:30.511505 9314 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:11:30.511637 9314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:11:30.511791 9314 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.129
I20260811 19:11:30.523607 9314 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:39965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab
--principal=kudu/127.3.61.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.61.129
--webserver_port=0
--tserver_master_addrs=127.3.61.190:33103
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.129
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:11:30.525133 9314 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:11:30.526645 9314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:11:30.542277 9330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:30.542328 9320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:30.542302 9322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:11:30.543134 9314 server_base.cc:1034] running on GCE node
I20260811 19:11:30.543819 9314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:11:30.545018 9314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:11:30.546262 9314 hybrid_clock.cc:648] HybridClock initialized: now 1786475490546197 us; error 58 us; skew 500 ppm
I20260811 19:11:30.546646 9314 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:11:30 dist-test-slave-t7g5 krb5kdc[9096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475490, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:11:30.551316 9314 init.cc:377] Logged in from keytab as kudu/127.3.61.129@KRBTEST.COM (short username kudu)
I20260811 19:11:30.553052 9314 webserver.cc:460] Webserver started at http://127.3.61.129:33349/ using document root <none> and password file <none>
I20260811 19:11:30.553902 9314 fs_manager.cc:359] Metadata directory not provided
I20260811 19:11:30.554044 9314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:11:30.554379 9314 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:11:30.558482 9314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-0/data/instance:
uuid: "bada5b5ad1a344e28a5bf05ed695a394"
format_stamp: "Formatted at 2026-08-11 19:11:30 on dist-test-slave-t7g5"
server_key: "538657eb7b043e65d845a01901ead145"
server_key_iv: "42a3b9c81cdbe7d4e207c04f2d055638"
server_key_version: "encryptionkey@0"
I20260811 19:11:30.559652 9314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-0/wal/instance:
uuid: "bada5b5ad1a344e28a5bf05ed695a394"
format_stamp: "Formatted at 2026-08-11 19:11:30 on dist-test-slave-t7g5"
server_key: "538657eb7b043e65d845a01901ead145"
server_key_iv: "42a3b9c81cdbe7d4e207c04f2d055638"
server_key_version: "encryptionkey@0"
I20260811 19:11:30.567301 9314 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.004s
I20260811 19:11:30.572271 9344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:30.574608 9314 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260811 19:11:30.574942 9314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-0/wal
uuid: "bada5b5ad1a344e28a5bf05ed695a394"
format_stamp: "Formatted at 2026-08-11 19:11:30 on dist-test-slave-t7g5"
server_key: "538657eb7b043e65d845a01901ead145"
server_key_iv: "42a3b9c81cdbe7d4e207c04f2d055638"
server_key_version: "encryptionkey@0"
I20260811 19:11:30.575258 9314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:11:30.598767 9314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:11:30.603629 9314 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:11:30.603965 9314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:11:30.605033 9314 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:11:30.606526 9314 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:11:30.606608 9314 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:30.606738 9314 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:11:30.606794 9314 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:30.643784 9314 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.129:46161
I20260811 19:11:30.643888 9571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.129:46161 every 8 connection(s)
I20260811 19:11:30.645195 9314 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
I20260811 19:11:30.649389 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 9314
I20260811 19:11:30.649633 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-0/wal/instance
I20260811 19:11:30.650422 3318 external_mini_cluster.cc:1428] Setting key 79ac7dc1512e144ff26f8a332bc0fb6f
Aug 11 19:11:30 dist-test-slave-t7g5 krb5kdc[9096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475490, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.129@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:11:30.672629 9576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:30.647945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:35253 (local address 127.3.61.190:33103)
0811 19:11:30.648475 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:30.648482 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:30.649903 (+ 1421us) server_negotiation.cc:407] Connection header received
0811 19:11:30.652846 (+ 2943us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:30.652851 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:30.652942 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:30.653038 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:30.655541 (+ 2503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:30.656311 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:30.658686 (+ 2375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:30.659126 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:30.659241 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:30.664716 (+ 5475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:30.664723 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:30.664726 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:30.664773 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:30.667525 (+ 2752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:30.669079 (+ 1554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:30.669086 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:30.669089 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:30.669292 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:30.670161 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:30.670166 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:30.670169 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:30.670603 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:30.670776 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:30.671990 (+ 1214us) server_negotiation.cc:299] Negotiation successful
0811 19:11:30.672282 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":256,"threads_started":1}
I20260811 19:11:30.674214 9578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:30.648457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33103 (local address 127.3.61.129:35253)
0811 19:11:30.649594 (+ 1137us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:30.649712 (+ 118us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:30.652252 (+ 2540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:30.653399 (+ 1147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:30.653413 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:30.654248 (+ 835us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:11:30.655296 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:30.655327 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:30.656486 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:30.656492 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:30.658444 (+ 1952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:30.658461 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:30.659385 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:30.659392 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:30.659694 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:30.660473 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:11:30.660509 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:11:30.664274 (+ 3765us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:11:30.667795 (+ 3521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:30.667811 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:30.667839 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:30.668793 (+ 954us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:30.669485 (+ 692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:30.669492 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:30.669496 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:30.669955 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:30.670774 (+ 819us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:11:30.670784 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:11:30.671408 (+ 624us) client_negotiation.cc:769] Sending connection context
0811 19:11:30.671810 (+ 402us) client_negotiation.cc:240] Negotiation successful
0811 19:11:30.672497 (+ 687us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":686,"thread_start_us":402,"threads_started":1}
I20260811 19:11:30.676385 9573 heartbeater.cc:347] Connected to a master server at 127.3.61.190:33103
I20260811 19:11:30.676752 9573 heartbeater.cc:464] Registering TS with master...
I20260811 19:11:30.677759 9573 heartbeater.cc:511] Master 127.3.61.190:33103 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.3.61.130@KRBTEST.COM; defaulting to no policy
I20260811 19:11:30.680382 9187 ts_manager.cc:194] Registered new tserver with Master: bada5b5ad1a344e28a5bf05ed695a394 (127.3.61.129:46161)
I20260811 19:11:30.682060 9187 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.129@KRBTEST.COM'} at 127.3.61.129:35253
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab.
Entry for principal kudu/127.3.61.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab.
I20260811 19:11:30.713135 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.130:0
--local_ip_for_outbound_sockets=127.3.61.130
--webserver_interface=127.3.61.130
--webserver_port=0
--tserver_master_addrs=127.3.61.190:33103
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab
--principal=kudu/127.3.61.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:39965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:11:31.090842 9582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:11:31.091169 9582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:11:31.091267 9582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:11:31.099247 9582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:11:31.099406 9582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:11:31.099541 9582 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.130
I20260811 19:11:31.109923 9582 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:39965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab
--principal=kudu/127.3.61.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.61.130
--webserver_port=0
--tserver_master_addrs=127.3.61.190:33103
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.130
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:11:31.111279 9582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:11:31.112658 9582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:11:31.127091 9591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:31.127130 9595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:31.127177 9588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:11:31.128118 9582 server_base.cc:1034] running on GCE node
I20260811 19:11:31.129014 9582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:11:31.130337 9582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:11:31.131791 9582 hybrid_clock.cc:648] HybridClock initialized: now 1786475491131733 us; error 45 us; skew 500 ppm
I20260811 19:11:31.132150 9582 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:11:31 dist-test-slave-t7g5 krb5kdc[9096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475491, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:11:31.135759 9582 init.cc:377] Logged in from keytab as kudu/127.3.61.130@KRBTEST.COM (short username kudu)
I20260811 19:11:31.137933 9582 webserver.cc:460] Webserver started at http://127.3.61.130:45973/ using document root <none> and password file <none>
I20260811 19:11:31.138681 9582 fs_manager.cc:359] Metadata directory not provided
I20260811 19:11:31.138821 9582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:11:31.139168 9582 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:11:31.142498 9582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-1/data/instance:
uuid: "9354d565358c49829c30052a31e1bb09"
format_stamp: "Formatted at 2026-08-11 19:11:31 on dist-test-slave-t7g5"
server_key: "cc2ea8c8ed9cea4caa475bafbf0ff3bc"
server_key_iv: "72e776ad5bedc14e1a05aab4ff6c9ffc"
server_key_version: "encryptionkey@0"
I20260811 19:11:31.143474 9582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-1/wal/instance:
uuid: "9354d565358c49829c30052a31e1bb09"
format_stamp: "Formatted at 2026-08-11 19:11:31 on dist-test-slave-t7g5"
server_key: "cc2ea8c8ed9cea4caa475bafbf0ff3bc"
server_key_iv: "72e776ad5bedc14e1a05aab4ff6c9ffc"
server_key_version: "encryptionkey@0"
I20260811 19:11:31.151602 9582 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.008s sys 0.000s
I20260811 19:11:31.156502 9605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:31.158489 9582 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 19:11:31.158689 9582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-1/wal
uuid: "9354d565358c49829c30052a31e1bb09"
format_stamp: "Formatted at 2026-08-11 19:11:31 on dist-test-slave-t7g5"
server_key: "cc2ea8c8ed9cea4caa475bafbf0ff3bc"
server_key_iv: "72e776ad5bedc14e1a05aab4ff6c9ffc"
server_key_version: "encryptionkey@0"
I20260811 19:11:31.158932 9582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:11:31.185189 9582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:11:31.190749 9582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:11:31.191208 9582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:11:31.193275 9582 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:11:31.195497 9582 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:11:31.195619 9582 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:31.195811 9582 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:11:31.195901 9582 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:31.240888 9582 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.130:37749
I20260811 19:11:31.240890 9827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.130:37749 every 8 connection(s)
I20260811 19:11:31.242746 9582 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
I20260811 19:11:31.244478 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 9582
I20260811 19:11:31.244691 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-1/wal/instance
I20260811 19:11:31.245260 3318 external_mini_cluster.cc:1428] Setting key e60482e2c7b6c066806d71859525d996
Aug 11 19:11:31 dist-test-slave-t7g5 krb5kdc[9096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475491, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.130@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:11:31.266878 9831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:31.246325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:50707 (local address 127.3.61.190:33103)
0811 19:11:31.247074 (+ 749us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:31.247083 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:31.248475 (+ 1392us) server_negotiation.cc:407] Connection header received
0811 19:11:31.250765 (+ 2290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:31.250771 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:31.250873 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:31.250971 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:31.252680 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:31.253417 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:31.255092 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:31.255652 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:31.255797 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:31.259858 (+ 4061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:31.259865 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:31.259869 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:31.259907 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:31.262934 (+ 3027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:31.264022 (+ 1088us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:31.264027 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:31.264029 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:31.264213 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:31.264855 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:31.264859 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:31.264862 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:31.265200 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:31.265349 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:31.266371 (+ 1022us) server_negotiation.cc:299] Negotiation successful
0811 19:11:31.266602 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":391,"threads_started":1}
I20260811 19:11:31.268419 9832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:31.247224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33103 (local address 127.3.61.130:50707)
0811 19:11:31.248276 (+ 1052us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:31.248348 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:31.250438 (+ 2090us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:31.251260 (+ 822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:31.251270 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:31.251861 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:11:31.252495 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:31.252514 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:31.253568 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:31.253572 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:31.254934 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:31.254949 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:31.255934 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:31.255942 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:31.256185 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:31.257007 (+ 822us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:11:31.257032 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:11:31.259566 (+ 2534us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:11:31.263158 (+ 3592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:31.263168 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:31.263188 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:31.263859 (+ 671us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:31.264360 (+ 501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:31.264365 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:31.264368 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:31.264696 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:31.265365 (+ 669us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:11:31.265373 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:11:31.265873 (+ 500us) client_negotiation.cc:769] Sending connection context
0811 19:11:31.266241 (+ 368us) client_negotiation.cc:240] Negotiation successful
0811 19:11:31.266861 (+ 620us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":726,"thread_start_us":398,"threads_started":1}
I20260811 19:11:31.270081 9829 heartbeater.cc:347] Connected to a master server at 127.3.61.190:33103
I20260811 19:11:31.270416 9829 heartbeater.cc:464] Registering TS with master...
I20260811 19:11:31.271201 9829 heartbeater.cc:511] Master 127.3.61.190:33103 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.3.61.131@KRBTEST.COM; defaulting to no policy
I20260811 19:11:31.272804 9187 ts_manager.cc:194] Registered new tserver with Master: 9354d565358c49829c30052a31e1bb09 (127.3.61.130:37749)
I20260811 19:11:31.273682 9187 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.130@KRBTEST.COM'} at 127.3.61.130:50707
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab.
Entry for principal kudu/127.3.61.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab.
I20260811 19:11:31.309199 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.131:0
--local_ip_for_outbound_sockets=127.3.61.131
--webserver_interface=127.3.61.131
--webserver_port=0
--tserver_master_addrs=127.3.61.190:33103
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab
--principal=kudu/127.3.61.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:39965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:11:31.685519 9573 heartbeater.cc:503] Master 127.3.61.190:33103 was elected leader, sending a full tablet report...
W20260811 19:11:31.688480 9838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:11:31.688792 9838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:11:31.688925 9838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:11:31.696710 9838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:11:31.697084 9838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:11:31.697362 9838 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.131
I20260811 19:11:31.709088 9838 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:39965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab
--principal=kudu/127.3.61.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.61.131
--webserver_port=0
--tserver_master_addrs=127.3.61.190:33103
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.131
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:11:31.710652 9838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:11:31.712083 9838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:11:31.726279 9849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:31.726285 9857 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:31.726325 9853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:11:31.726987 9838 server_base.cc:1034] running on GCE node
I20260811 19:11:31.727723 9838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:11:31.728768 9838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:11:31.729974 9838 hybrid_clock.cc:648] HybridClock initialized: now 1786475491729912 us; error 52 us; skew 500 ppm
I20260811 19:11:31.730296 9838 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:11:31 dist-test-slave-t7g5 krb5kdc[9096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475491, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:11:31.733637 9838 init.cc:377] Logged in from keytab as kudu/127.3.61.131@KRBTEST.COM (short username kudu)
I20260811 19:11:31.735363 9838 webserver.cc:460] Webserver started at http://127.3.61.131:36381/ using document root <none> and password file <none>
I20260811 19:11:31.736099 9838 fs_manager.cc:359] Metadata directory not provided
I20260811 19:11:31.736212 9838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:11:31.736498 9838 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:11:31.739691 9838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-2/data/instance:
uuid: "28f18f776cb740b6beff50f4d63cff13"
format_stamp: "Formatted at 2026-08-11 19:11:31 on dist-test-slave-t7g5"
server_key: "2725e20f18551b7f9ed561f0af372034"
server_key_iv: "58b3f79af5a155133294624f4c3a3e71"
server_key_version: "encryptionkey@0"
I20260811 19:11:31.740499 9838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-2/wal/instance:
uuid: "28f18f776cb740b6beff50f4d63cff13"
format_stamp: "Formatted at 2026-08-11 19:11:31 on dist-test-slave-t7g5"
server_key: "2725e20f18551b7f9ed561f0af372034"
server_key_iv: "58b3f79af5a155133294624f4c3a3e71"
server_key_version: "encryptionkey@0"
I20260811 19:11:31.748377 9838 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.004s
I20260811 19:11:31.753152 9874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:31.754878 9838 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 19:11:31.755105 9838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-2/wal
uuid: "28f18f776cb740b6beff50f4d63cff13"
format_stamp: "Formatted at 2026-08-11 19:11:31 on dist-test-slave-t7g5"
server_key: "2725e20f18551b7f9ed561f0af372034"
server_key_iv: "58b3f79af5a155133294624f4c3a3e71"
server_key_version: "encryptionkey@0"
I20260811 19:11:31.755340 9838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:11:31.773968 9838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:11:31.779388 9838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:11:31.779718 9838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:11:31.781277 9838 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:11:31.783313 9838 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:11:31.783413 9838 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:31.783576 9838 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:11:31.783655 9838 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:31.822997 9838 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.131:36783
I20260811 19:11:31.823132 10100 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.131:36783 every 8 connection(s)
I20260811 19:11:31.824563 9838 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
I20260811 19:11:31.831249 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 9838
I20260811 19:11:31.831490 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786475454804547-3318-0/minicluster-data/ts-2/wal/instance
I20260811 19:11:31.832226 3318 external_mini_cluster.cc:1428] Setting key 0d0fc825327f3155b4ff4bda851d0a1e
Aug 11 19:11:31 dist-test-slave-t7g5 krb5kdc[9096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475491, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.131@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:11:31.851693 10106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:31.828351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:57621 (local address 127.3.61.190:33103)
0811 19:11:31.829036 (+ 685us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:31.829044 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:31.830373 (+ 1329us) server_negotiation.cc:407] Connection header received
0811 19:11:31.832873 (+ 2500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:31.832879 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:31.832974 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:31.833095 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:31.835268 (+ 2173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:31.836316 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:31.838135 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:31.838571 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:31.838699 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:31.843989 (+ 5290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:31.843996 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:31.843999 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:31.844041 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:31.847728 (+ 3687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:31.848814 (+ 1086us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:31.848821 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:31.848823 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:31.848972 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:31.849699 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:31.849707 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:31.849710 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:31.850051 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:31.850219 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:31.851029 (+ 810us) server_negotiation.cc:299] Negotiation successful
0811 19:11:31.851313 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":497,"thread_start_us":325,"threads_started":1}
I20260811 19:11:31.852842 10107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:31.829151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33103 (local address 127.3.61.131:57621)
0811 19:11:31.830174 (+ 1023us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:31.830234 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:31.832480 (+ 2246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:31.833474 (+ 994us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:31.833488 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:31.834328 (+ 840us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:11:31.835088 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:31.835110 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:31.836514 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:31.836519 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:31.837954 (+ 1435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:31.837970 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:31.838851 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:31.838896 (+ 45us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:31.839248 (+ 352us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:31.840081 (+ 833us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:11:31.840128 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:11:31.843670 (+ 3542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:11:31.847980 (+ 4310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:31.847993 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:31.848017 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:31.848654 (+ 637us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:31.849187 (+ 533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:31.849193 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:31.849196 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:31.849498 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:31.850202 (+ 704us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:11:31.850213 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:11:31.850609 (+ 396us) client_negotiation.cc:769] Sending connection context
0811 19:11:31.850882 (+ 273us) client_negotiation.cc:240] Negotiation successful
0811 19:11:31.851491 (+ 609us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":724,"thread_start_us":465,"threads_started":1}
I20260811 19:11:31.854426 10102 heartbeater.cc:347] Connected to a master server at 127.3.61.190:33103
I20260811 19:11:31.854770 10102 heartbeater.cc:464] Registering TS with master...
I20260811 19:11:31.855636 10102 heartbeater.cc:511] Master 127.3.61.190:33103 requested a full tablet report, sending...
I20260811 19:11:31.857718 9187 ts_manager.cc:194] Registered new tserver with Master: 28f18f776cb740b6beff50f4d63cff13 (127.3.61.131:36783)
I20260811 19:11:31.859476 9187 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.131@KRBTEST.COM'} at 127.3.61.131:57621
I20260811 19:11:31.861956 3318 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 19:11:31 dist-test-slave-t7g5 krb5kdc[9096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475491, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 11 19:11:31 dist-test-slave-t7g5 krb5kdc[9096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475491, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:11:31.928375 10106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:31.910924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53276 (local address 127.3.61.190:33103)
0811 19:11:31.911203 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:31.911212 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:31.911609 (+ 397us) server_negotiation.cc:407] Connection header received
0811 19:11:31.911855 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:31.911860 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:31.911958 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:31.912061 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:31.913741 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:31.914780 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:31.916318 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:31.916757 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:31.916881 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:31.921999 (+ 5118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:31.922005 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:31.922009 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:31.922065 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:31.925503 (+ 3438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:31.926334 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:31.926339 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:31.926342 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:31.926484 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:31.926871 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:31.926875 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:31.926877 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:31.927262 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:31.927546 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:31.927830 (+ 284us) server_negotiation.cc:299] Negotiation successful
0811 19:11:31.928069 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
W20260811 19:11:31.928953 9187 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:53276
I20260811 19:11:31.931476 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 9314
Aug 11 19:11:31 dist-test-slave-t7g5 krb5kdc[9096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475491, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.3.61.129@KRBTEST.COM
I20260811 19:11:31.950235 10126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:31.932468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59888 (local address 127.3.61.129:46161)
0811 19:11:31.933076 (+ 608us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:31.933083 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:31.933111 (+ 28us) server_negotiation.cc:407] Connection header received
0811 19:11:31.933173 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:31.933180 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:31.933410 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:31.933541 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:31.934858 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:31.935930 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:31.937283 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:31.938382 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:31.938642 (+ 260us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:31.943141 (+ 4499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:31.943157 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:31.943174 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:31.943228 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:31.946507 (+ 3279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:31.947304 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:31.947312 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:31.947314 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:31.947467 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:31.948175 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:31.948180 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:31.948183 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:31.948590 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:31.949078 (+ 488us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:31.949293 (+ 215us) server_negotiation.cc:299] Negotiation successful
0811 19:11:31.949828 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":276,"threads_started":1}
W20260811 19:11:31.951892 9564 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:59888
W20260811 19:11:31.953225 3318 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 9314: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 19:11:31.953447 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 9314
I20260811 19:11:31.980226 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 9582
Aug 11 19:11:31 dist-test-slave-t7g5 krb5kdc[9096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475491, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.3.61.130@KRBTEST.COM
I20260811 19:11:32.000378 10127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:31.981567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33502 (local address 127.3.61.130:37749)
0811 19:11:31.982184 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:31.982192 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:31.982224 (+ 32us) server_negotiation.cc:407] Connection header received
0811 19:11:31.982295 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:31.982302 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:31.982555 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:31.982690 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:31.984058 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:31.985468 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:31.987145 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:31.987816 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:31.987962 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:31.993127 (+ 5165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:31.993152 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:31.993176 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:31.993264 (+ 88us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:31.996641 (+ 3377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:31.997555 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:31.997565 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:31.997567 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:31.997776 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:31.998235 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:31.998239 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:31.998241 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:31.998646 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:31.998900 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:31.999488 (+ 588us) server_negotiation.cc:299] Negotiation successful
0811 19:11:31.999981 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":308,"threads_started":1}
W20260811 19:11:32.002158 9819 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:33502
W20260811 19:11:32.003818 3318 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 9582: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 19:11:32.004081 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 9582
I20260811 19:11:32.033279 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 9838
Aug 11 19:11:32 dist-test-slave-t7g5 krb5kdc[9096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475491, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.3.61.131@KRBTEST.COM
I20260811 19:11:32.055940 10131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:32.035148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42126 (local address 127.3.61.131:36783)
0811 19:11:32.036027 (+ 879us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:32.036035 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:32.036071 (+ 36us) server_negotiation.cc:407] Connection header received
0811 19:11:32.036148 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:32.036155 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:32.036386 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:32.036552 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:32.038540 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:32.039905 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:32.041533 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:32.042269 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:32.042391 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:32.048261 (+ 5870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:32.048289 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:32.048315 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:32.048380 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:32.052229 (+ 3849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:32.053296 (+ 1067us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:32.053308 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:32.053311 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:32.053572 (+ 261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:32.054088 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:32.054093 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:32.054096 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:32.054577 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:32.054870 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:32.055249 (+ 379us) server_negotiation.cc:299] Negotiation successful
0811 19:11:32.055549 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":672,"thread_start_us":456,"threads_started":1}
W20260811 19:11:32.057518 10095 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:42126
W20260811 19:11:32.058919 3318 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 9838: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 19:11:32.059103 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 9838
I20260811 19:11:32.084692 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 9119
I20260811 19:11:32.086026 9256 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:11:32.207319 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 9119
2026-08-11T19:11:32Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (4401 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:11:32 dist-test-slave-t7g5 krb5kdc[10141](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:11:32 dist-test-slave-t7g5 krb5kdc[10141](info): set up 2 sockets
Aug 11 19:11:32 dist-test-slave-t7g5 krb5kdc[10141](info): commencing operation
krb5kdc: starting...
W20260811 19:11:33.524122 3318 mini_kdc.cc:121] Time spent starting KDC: real 1.259s user 0.004s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Aug 11 19:11:33 dist-test-slave-t7g5 krb5kdc[10141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475493, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:11:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:11:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.61.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab.
Entry for principal kudu/127.3.61.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab.
I20260811 19:11:33.802294 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:37267
--webserver_interface=127.3.61.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:40811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.61.190:37267
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:11:34.164557 10162 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:11:34.164927 10162 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:11:34.165051 10162 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:11:34.172518 10162 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:11:34.172649 10162 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:11:34.172708 10162 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:11:34.172791 10162 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:11:34.172856 10162 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:11:34.183678 10162 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:40811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.61.190:37267
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:37267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.61.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:11:34.185307 10162 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:11:34.186956 10162 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:11:34.203794 10175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:34.203823 10173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:34.204241 10179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:11:34.204840 10162 server_base.cc:1034] running on GCE node
I20260811 19:11:34.205977 10162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:11:34.207553 10162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:11:34.208808 10162 hybrid_clock.cc:648] HybridClock initialized: now 1786475494208752 us; error 36 us; skew 500 ppm
I20260811 19:11:34.209153 10162 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:11:34 dist-test-slave-t7g5 krb5kdc[10141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475494, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:11:34.212931 10162 init.cc:377] Logged in from keytab as kudu/127.3.61.190@KRBTEST.COM (short username kudu)
I20260811 19:11:34.214650 10162 webserver.cc:460] Webserver started at http://127.3.61.190:39027/ using document root <none> and password file <none>
I20260811 19:11:34.215380 10162 fs_manager.cc:359] Metadata directory not provided
I20260811 19:11:34.215507 10162 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:11:34.215968 10162 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:11:34.219177 10162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/master-0/data/instance:
uuid: "62d004bd6927499898a1ff14037587e0"
format_stamp: "Formatted at 2026-08-11 19:11:34 on dist-test-slave-t7g5"
server_key: "cdc8684095f8ba7ea36ce083f9ab45a3"
server_key_iv: "dca07e473490ffa78b81c799ff2d91ca"
server_key_version: "encryptionkey@0"
I20260811 19:11:34.220117 10162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/master-0/wal/instance:
uuid: "62d004bd6927499898a1ff14037587e0"
format_stamp: "Formatted at 2026-08-11 19:11:34 on dist-test-slave-t7g5"
server_key: "cdc8684095f8ba7ea36ce083f9ab45a3"
server_key_iv: "dca07e473490ffa78b81c799ff2d91ca"
server_key_version: "encryptionkey@0"
I20260811 19:11:34.225632 10162 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 19:11:34.229488 10189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:34.231280 10162 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260811 19:11:34.231468 10162 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/master-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/master-0/wal
uuid: "62d004bd6927499898a1ff14037587e0"
format_stamp: "Formatted at 2026-08-11 19:11:34 on dist-test-slave-t7g5"
server_key: "cdc8684095f8ba7ea36ce083f9ab45a3"
server_key_iv: "dca07e473490ffa78b81c799ff2d91ca"
server_key_version: "encryptionkey@0"
I20260811 19:11:34.231705 10162 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:11:34.256371 10162 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:11:34.262259 10162 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:11:34.262674 10162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:11:34.283833 10162 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.190:37267
I20260811 19:11:34.283998 10271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.190:37267 every 8 connection(s)
I20260811 19:11:34.285463 10162 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
I20260811 19:11:34.289685 10279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:11:34.292459 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 10162
I20260811 19:11:34.292716 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/master-0/wal/instance
I20260811 19:11:34.293506 3318 external_mini_cluster.cc:1428] Setting key e7e2426abfd290548946caa9d3816f89
I20260811 19:11:34.304476 10279 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62d004bd6927499898a1ff14037587e0: Bootstrap starting.
I20260811 19:11:34.308822 10279 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 62d004bd6927499898a1ff14037587e0: Neither blocks nor log segments found. Creating new log.
I20260811 19:11:34.310663 10279 log.cc:824] T 00000000000000000000000000000000 P 62d004bd6927499898a1ff14037587e0: Log is configured to *not* fsync() on all Append() calls
Aug 11 19:11:34 dist-test-slave-t7g5 krb5kdc[10141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475493, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:11:34.315385 10279 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62d004bd6927499898a1ff14037587e0: No bootstrap required, opened a new log
I20260811 19:11:34.324337 10287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:34.295979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36578 (local address 127.3.61.190:37267)
0811 19:11:34.296877 (+ 898us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:34.296890 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:34.296938 (+ 48us) server_negotiation.cc:407] Connection header received
0811 19:11:34.298823 (+ 1885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:34.298859 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:34.299310 (+ 451us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:34.299696 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:34.301776 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:34.303357 (+ 1581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:34.305555 (+ 2198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:34.306611 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:34.306852 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:34.313021 (+ 6169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:34.313036 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:34.313061 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:34.313115 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:34.318124 (+ 5009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:34.319356 (+ 1232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:34.319367 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:34.319379 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:34.319687 (+ 308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:34.320219 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:34.320224 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:34.320227 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:34.321116 (+ 889us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:34.321509 (+ 393us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:34.322129 (+ 620us) server_negotiation.cc:299] Negotiation successful
0811 19:11:34.322736 (+ 607us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":664,"thread_start_us":413,"threads_started":1}
I20260811 19:11:34.326561 10279 raft_consensus.cc:348] T 00000000000000000000000000000000 P 62d004bd6927499898a1ff14037587e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62d004bd6927499898a1ff14037587e0" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 37267 } }
I20260811 19:11:34.326978 10279 raft_consensus.cc:374] T 00000000000000000000000000000000 P 62d004bd6927499898a1ff14037587e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:11:34.327093 10279 raft_consensus.cc:729] T 00000000000000000000000000000000 P 62d004bd6927499898a1ff14037587e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62d004bd6927499898a1ff14037587e0, State: Initialized, Role: FOLLOWER
I20260811 19:11:34.327833 10279 consensus_queue.cc:260] T 00000000000000000000000000000000 P 62d004bd6927499898a1ff14037587e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62d004bd6927499898a1ff14037587e0" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 37267 } }
I20260811 19:11:34.328110 10279 raft_consensus.cc:388] T 00000000000000000000000000000000 P 62d004bd6927499898a1ff14037587e0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:11:34.328258 10279 raft_consensus.cc:482] T 00000000000000000000000000000000 P 62d004bd6927499898a1ff14037587e0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:11:34.328472 10279 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 62d004bd6927499898a1ff14037587e0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:11:34.330920 10279 raft_consensus.cc:504] T 00000000000000000000000000000000 P 62d004bd6927499898a1ff14037587e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62d004bd6927499898a1ff14037587e0" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 37267 } }
I20260811 19:11:34.331542 10279 leader_election.cc:302] T 00000000000000000000000000000000 P 62d004bd6927499898a1ff14037587e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 62d004bd6927499898a1ff14037587e0; no voters:
I20260811 19:11:34.332231 10279 leader_election.cc:288] T 00000000000000000000000000000000 P 62d004bd6927499898a1ff14037587e0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:11:34.332353 10289 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 62d004bd6927499898a1ff14037587e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:11:34.333012 10289 raft_consensus.cc:686] T 00000000000000000000000000000000 P 62d004bd6927499898a1ff14037587e0 [term 1 LEADER]: Becoming Leader. State: Replica: 62d004bd6927499898a1ff14037587e0, State: Running, Role: LEADER
I20260811 19:11:34.333640 10289 consensus_queue.cc:237] T 00000000000000000000000000000000 P 62d004bd6927499898a1ff14037587e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62d004bd6927499898a1ff14037587e0" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 37267 } }
I20260811 19:11:34.334843 10279 sys_catalog.cc:564] T 00000000000000000000000000000000 P 62d004bd6927499898a1ff14037587e0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:11:34.336838 10291 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62d004bd6927499898a1ff14037587e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 62d004bd6927499898a1ff14037587e0. Latest consensus state: current_term: 1 leader_uuid: "62d004bd6927499898a1ff14037587e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62d004bd6927499898a1ff14037587e0" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 37267 } } }
I20260811 19:11:34.337510 10291 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62d004bd6927499898a1ff14037587e0 [sys.catalog]: This master's current role is: LEADER
I20260811 19:11:34.336865 10290 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62d004bd6927499898a1ff14037587e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "62d004bd6927499898a1ff14037587e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62d004bd6927499898a1ff14037587e0" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 37267 } } }
I20260811 19:11:34.337833 10290 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62d004bd6927499898a1ff14037587e0 [sys.catalog]: This master's current role is: LEADER
I20260811 19:11:34.338608 10305 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:11:34.346452 10305 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:11:34.357852 10305 catalog_manager.cc:1347] Generated new cluster ID: 2b92e2aea78c403d9e619e154fed5b2d
I20260811 19:11:34.357996 10305 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:11:34.374224 10305 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:11:34.375725 10305 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:11:34.388137 10305 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 62d004bd6927499898a1ff14037587e0: Generated new TSK 0
I20260811 19:11:34.389202 10305 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.61.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab.
Entry for principal kudu/127.3.61.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab.
I20260811 19:11:34.458531 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.129:0
--local_ip_for_outbound_sockets=127.3.61.129
--webserver_interface=127.3.61.129
--webserver_port=0
--tserver_master_addrs=127.3.61.190:37267
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab
--principal=kudu/127.3.61.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:40811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:11:34.833226 10325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:11:34.833591 10325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:11:34.833768 10325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:11:34.841977 10325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:11:34.842147 10325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:11:34.842461 10325 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.129
I20260811 19:11:34.854580 10325 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:40811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab
--principal=kudu/127.3.61.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.61.129
--webserver_port=0
--tserver_master_addrs=127.3.61.190:37267
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.129
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:11:34.856022 10325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:11:34.857452 10325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:11:34.871613 10341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:34.871753 10339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:34.871675 10343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:11:34.873085 10325 server_base.cc:1034] running on GCE node
I20260811 19:11:34.873929 10325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:11:34.875133 10325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:11:34.876536 10325 hybrid_clock.cc:648] HybridClock initialized: now 1786475494876471 us; error 100 us; skew 500 ppm
I20260811 19:11:34.876935 10325 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:11:34 dist-test-slave-t7g5 krb5kdc[10141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475494, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:11:34.884025 10325 init.cc:377] Logged in from keytab as kudu/127.3.61.129@KRBTEST.COM (short username kudu)
I20260811 19:11:34.885792 10325 webserver.cc:460] Webserver started at http://127.3.61.129:39771/ using document root <none> and password file <none>
I20260811 19:11:34.886531 10325 fs_manager.cc:359] Metadata directory not provided
I20260811 19:11:34.886646 10325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:11:34.887022 10325 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:11:34.890336 10325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/ts-0/data/instance:
uuid: "37fb94b8eb9041b8a815cacf57660ca5"
format_stamp: "Formatted at 2026-08-11 19:11:34 on dist-test-slave-t7g5"
server_key: "8e49b057ed557d32ce0dd545ae30ff69"
server_key_iv: "0d38494a540d99c5064fd6bd7e83caf7"
server_key_version: "encryptionkey@0"
I20260811 19:11:34.891278 10325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/ts-0/wal/instance:
uuid: "37fb94b8eb9041b8a815cacf57660ca5"
format_stamp: "Formatted at 2026-08-11 19:11:34 on dist-test-slave-t7g5"
server_key: "8e49b057ed557d32ce0dd545ae30ff69"
server_key_iv: "0d38494a540d99c5064fd6bd7e83caf7"
server_key_version: "encryptionkey@0"
I20260811 19:11:34.897243 10325 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260811 19:11:34.901865 10357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:34.903712 10325 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 19:11:34.903918 10325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/ts-0/wal
uuid: "37fb94b8eb9041b8a815cacf57660ca5"
format_stamp: "Formatted at 2026-08-11 19:11:34 on dist-test-slave-t7g5"
server_key: "8e49b057ed557d32ce0dd545ae30ff69"
server_key_iv: "0d38494a540d99c5064fd6bd7e83caf7"
server_key_version: "encryptionkey@0"
I20260811 19:11:34.904167 10325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:11:34.937274 10325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:11:34.943030 10325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:11:34.943358 10325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:11:34.944568 10325 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:11:34.946205 10325 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:11:34.946297 10325 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:34.946420 10325 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:11:34.946484 10325 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:34.982369 10325 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.129:45141
I20260811 19:11:34.982442 10574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.129:45141 every 8 connection(s)
I20260811 19:11:34.983794 10325 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
I20260811 19:11:34.990520 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 10325
I20260811 19:11:34.990810 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/ts-0/wal/instance
I20260811 19:11:34.991788 3318 external_mini_cluster.cc:1428] Setting key a4639a7dc77f5718e427ff6f841ad543
Aug 11 19:11:34 dist-test-slave-t7g5 krb5kdc[10141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475494, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.129@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:11:35.011320 10580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:34.986787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:36325 (local address 127.3.61.190:37267)
0811 19:11:34.987312 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:34.987317 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:34.988775 (+ 1458us) server_negotiation.cc:407] Connection header received
0811 19:11:34.991403 (+ 2628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:34.991410 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:34.991521 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:34.991635 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:34.994152 (+ 2517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:34.995233 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:34.997266 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:34.997954 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:34.998126 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:35.002744 (+ 4618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:35.002752 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:35.002757 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:35.002804 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:35.006175 (+ 3371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:35.007912 (+ 1737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:35.007920 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:35.007922 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:35.008131 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:35.009106 (+ 975us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:35.009112 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:35.009114 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:35.009507 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:35.009700 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:35.010803 (+ 1103us) server_negotiation.cc:299] Negotiation successful
0811 19:11:35.011051 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":264,"threads_started":1}
I20260811 19:11:35.012873 10581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:34.987374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:37267 (local address 127.3.61.129:36325)
0811 19:11:34.988447 (+ 1073us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:34.988559 (+ 112us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:34.991009 (+ 2450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:34.992011 (+ 1002us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:34.992023 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:34.992854 (+ 831us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:11:34.993879 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:34.993910 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:34.995495 (+ 1585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:34.995499 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:34.997080 (+ 1581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:34.997096 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:34.998347 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:34.998355 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:34.998659 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:34.999429 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:11:34.999457 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:11:35.002395 (+ 2938us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:11:35.006611 (+ 4216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:35.006628 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:35.006659 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:35.007586 (+ 927us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:35.008314 (+ 728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:35.008319 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:35.008322 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:35.008925 (+ 603us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:35.009715 (+ 790us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:11:35.009725 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:11:35.010275 (+ 550us) client_negotiation.cc:769] Sending connection context
0811 19:11:35.010632 (+ 357us) client_negotiation.cc:240] Negotiation successful
0811 19:11:35.011280 (+ 648us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":667,"thread_start_us":414,"threads_started":1}
I20260811 19:11:35.014904 10576 heartbeater.cc:347] Connected to a master server at 127.3.61.190:37267
I20260811 19:11:35.015264 10576 heartbeater.cc:464] Registering TS with master...
I20260811 19:11:35.016130 10576 heartbeater.cc:511] Master 127.3.61.190:37267 requested a full tablet report, sending...
I20260811 19:11:35.019591 10214 ts_manager.cc:194] Registered new tserver with Master: 37fb94b8eb9041b8a815cacf57660ca5 (127.3.61.129:45141)
I20260811 19:11:35.022226 10214 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.129@KRBTEST.COM'} at 127.3.61.129:36325
WARNING: no policy specified for kudu/127.3.61.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab.
Entry for principal kudu/127.3.61.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab.
I20260811 19:11:35.062947 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.130:0
--local_ip_for_outbound_sockets=127.3.61.130
--webserver_interface=127.3.61.130
--webserver_port=0
--tserver_master_addrs=127.3.61.190:37267
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab
--principal=kudu/127.3.61.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:40811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:11:35.440613 10585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:11:35.440932 10585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:11:35.441064 10585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:11:35.448663 10585 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:11:35.448808 10585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:11:35.448962 10585 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.130
I20260811 19:11:35.459343 10585 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:40811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab
--principal=kudu/127.3.61.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.61.130
--webserver_port=0
--tserver_master_addrs=127.3.61.190:37267
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.130
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:11:35.460791 10585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:11:35.463095 10585 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:11:35.478236 10594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:35.478196 10595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:35.478278 10597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:11:35.480033 10585 server_base.cc:1034] running on GCE node
I20260811 19:11:35.480835 10585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:11:35.481972 10585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:11:35.483307 10585 hybrid_clock.cc:648] HybridClock initialized: now 1786475495483202 us; error 54 us; skew 500 ppm
I20260811 19:11:35.483907 10585 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:11:35 dist-test-slave-t7g5 krb5kdc[10141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475495, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:11:35.488148 10585 init.cc:377] Logged in from keytab as kudu/127.3.61.130@KRBTEST.COM (short username kudu)
I20260811 19:11:35.490080 10585 webserver.cc:460] Webserver started at http://127.3.61.130:38975/ using document root <none> and password file <none>
I20260811 19:11:35.490876 10585 fs_manager.cc:359] Metadata directory not provided
I20260811 19:11:35.490998 10585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:11:35.491325 10585 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:11:35.494812 10585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/ts-1/data/instance:
uuid: "f6583ab870ac444faedcb07bdcfdd304"
format_stamp: "Formatted at 2026-08-11 19:11:35 on dist-test-slave-t7g5"
server_key: "8b37f2f9dc8f6e0343da04409bba72b2"
server_key_iv: "b18593d059bc0ee96e3a1b2ef27ecede"
server_key_version: "encryptionkey@0"
I20260811 19:11:35.495887 10585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/ts-1/wal/instance:
uuid: "f6583ab870ac444faedcb07bdcfdd304"
format_stamp: "Formatted at 2026-08-11 19:11:35 on dist-test-slave-t7g5"
server_key: "8b37f2f9dc8f6e0343da04409bba72b2"
server_key_iv: "b18593d059bc0ee96e3a1b2ef27ecede"
server_key_version: "encryptionkey@0"
I20260811 19:11:35.503683 10585 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260811 19:11:35.508148 10612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:35.510068 10585 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 19:11:35.510265 10585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/ts-1/wal
uuid: "f6583ab870ac444faedcb07bdcfdd304"
format_stamp: "Formatted at 2026-08-11 19:11:35 on dist-test-slave-t7g5"
server_key: "8b37f2f9dc8f6e0343da04409bba72b2"
server_key_iv: "b18593d059bc0ee96e3a1b2ef27ecede"
server_key_version: "encryptionkey@0"
I20260811 19:11:35.510504 10585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:11:35.530107 10585 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:11:35.535188 10585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:11:35.535530 10585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:11:35.536721 10585 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:11:35.538540 10585 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:11:35.538646 10585 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:35.538777 10585 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:11:35.538842 10585 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:35.576208 10585 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.130:35237
I20260811 19:11:35.576243 10838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.130:35237 every 8 connection(s)
I20260811 19:11:35.577697 10585 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
I20260811 19:11:35.584414 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 10585
I20260811 19:11:35.584638 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/ts-1/wal/instance
I20260811 19:11:35.585109 3318 external_mini_cluster.cc:1428] Setting key a11dd8d3f6a5442969f02e6ab1905898
Aug 11 19:11:35 dist-test-slave-t7g5 krb5kdc[10141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475495, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.130@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:11:35.603214 10841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:35.581233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:53847 (local address 127.3.61.190:37267)
0811 19:11:35.581789 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:35.581794 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:35.583175 (+ 1381us) server_negotiation.cc:407] Connection header received
0811 19:11:35.585419 (+ 2244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:35.585424 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:35.585504 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:35.585605 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:35.587608 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:35.588342 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:35.590037 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:35.590465 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:35.590579 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:35.594651 (+ 4072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:35.594656 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:35.594660 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:35.594700 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:35.597943 (+ 3243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:35.599394 (+ 1451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:35.599400 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:35.599403 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:35.599604 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:35.600327 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:35.600333 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:35.600336 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:35.600749 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:35.600901 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:35.602489 (+ 1588us) server_negotiation.cc:299] Negotiation successful
0811 19:11:35.602867 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":259,"threads_started":1}
I20260811 19:11:35.604652 10843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:35.581841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:37267 (local address 127.3.61.130:53847)
0811 19:11:35.582870 (+ 1029us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:35.582930 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:35.584992 (+ 2062us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:35.585927 (+ 935us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:35.585938 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:35.586707 (+ 769us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:11:35.587423 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:35.587444 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:35.588506 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:35.588510 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:35.589881 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:35.589896 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:35.590705 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:35.590712 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:35.590952 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:35.591641 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:11:35.591668 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:11:35.594343 (+ 2675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:11:35.598234 (+ 3891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:35.598249 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:35.598279 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:35.599133 (+ 854us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:35.599751 (+ 618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:35.599757 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:35.599762 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:35.600078 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:35.600910 (+ 832us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:11:35.600922 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:11:35.601562 (+ 640us) client_negotiation.cc:769] Sending connection context
0811 19:11:35.602277 (+ 715us) client_negotiation.cc:240] Negotiation successful
0811 19:11:35.602983 (+ 706us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":681,"thread_start_us":389,"threads_started":1}
I20260811 19:11:35.606866 10839 heartbeater.cc:347] Connected to a master server at 127.3.61.190:37267
I20260811 19:11:35.607265 10839 heartbeater.cc:464] Registering TS with master...
I20260811 19:11:35.608268 10839 heartbeater.cc:511] Master 127.3.61.190:37267 requested a full tablet report, sending...
I20260811 19:11:35.610715 10214 ts_manager.cc:194] Registered new tserver with Master: f6583ab870ac444faedcb07bdcfdd304 (127.3.61.130:35237)
I20260811 19:11:35.612022 10214 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.130@KRBTEST.COM'} at 127.3.61.130:53847
WARNING: no policy specified for kudu/127.3.61.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab.
Entry for principal kudu/127.3.61.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab.
I20260811 19:11:35.653852 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.131:0
--local_ip_for_outbound_sockets=127.3.61.131
--webserver_interface=127.3.61.131
--webserver_port=0
--tserver_master_addrs=127.3.61.190:37267
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab
--principal=kudu/127.3.61.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:40811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:11:36.026970 10576 heartbeater.cc:503] Master 127.3.61.190:37267 was elected leader, sending a full tablet report...
W20260811 19:11:36.040365 10847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:11:36.041189 10847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:11:36.041384 10847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:11:36.050163 10847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:11:36.050359 10847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:11:36.050531 10847 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.131
I20260811 19:11:36.062233 10847 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:40811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab
--principal=kudu/127.3.61.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.61.131
--webserver_port=0
--tserver_master_addrs=127.3.61.190:37267
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.131
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:11:36.064213 10847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:11:36.066255 10847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:11:36.082612 10859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:36.082783 10855 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:36.082716 10854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:11:36.084045 10847 server_base.cc:1034] running on GCE node
I20260811 19:11:36.084818 10847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:11:36.085923 10847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:11:36.087210 10847 hybrid_clock.cc:648] HybridClock initialized: now 1786475496087147 us; error 46 us; skew 500 ppm
I20260811 19:11:36.087633 10847 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:11:36 dist-test-slave-t7g5 krb5kdc[10141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475496, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:11:36.103960 10847 init.cc:377] Logged in from keytab as kudu/127.3.61.131@KRBTEST.COM (short username kudu)
I20260811 19:11:36.106012 10847 webserver.cc:460] Webserver started at http://127.3.61.131:45929/ using document root <none> and password file <none>
I20260811 19:11:36.106842 10847 fs_manager.cc:359] Metadata directory not provided
I20260811 19:11:36.106981 10847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:11:36.107307 10847 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:11:36.111541 10847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/ts-2/data/instance:
uuid: "a68156b4a1cb44d2b66ea7511a4d304a"
format_stamp: "Formatted at 2026-08-11 19:11:36 on dist-test-slave-t7g5"
server_key: "c72ca0634f4aafca76df838be56ec0ea"
server_key_iv: "bf673f62761e01960a7cf37e49f0426a"
server_key_version: "encryptionkey@0"
I20260811 19:11:36.112686 10847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/ts-2/wal/instance:
uuid: "a68156b4a1cb44d2b66ea7511a4d304a"
format_stamp: "Formatted at 2026-08-11 19:11:36 on dist-test-slave-t7g5"
server_key: "c72ca0634f4aafca76df838be56ec0ea"
server_key_iv: "bf673f62761e01960a7cf37e49f0426a"
server_key_version: "encryptionkey@0"
I20260811 19:11:36.120002 10847 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.003s
I20260811 19:11:36.125358 10870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:36.127355 10847 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260811 19:11:36.127571 10847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/ts-2/wal
uuid: "a68156b4a1cb44d2b66ea7511a4d304a"
format_stamp: "Formatted at 2026-08-11 19:11:36 on dist-test-slave-t7g5"
server_key: "c72ca0634f4aafca76df838be56ec0ea"
server_key_iv: "bf673f62761e01960a7cf37e49f0426a"
server_key_version: "encryptionkey@0"
I20260811 19:11:36.127846 10847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:11:36.142252 10847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:11:36.147171 10847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:11:36.147491 10847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:11:36.148624 10847 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:11:36.150290 10847 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:11:36.150403 10847 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:36.150539 10847 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:11:36.150605 10847 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:36.188290 10847 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.131:35265
I20260811 19:11:36.188427 11089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.131:35265 every 8 connection(s)
I20260811 19:11:36.190050 10847 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
I20260811 19:11:36.195031 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 10847
I20260811 19:11:36.195262 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786475454804547-3318-0/minicluster-data/ts-2/wal/instance
I20260811 19:11:36.195863 3318 external_mini_cluster.cc:1428] Setting key ed068a49656085e05cf5a9a1cf44eac0
Aug 11 19:11:36 dist-test-slave-t7g5 krb5kdc[10141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475496, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.131@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:11:36.217949 11093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:36.195508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:45249 (local address 127.3.61.190:37267)
0811 19:11:36.196007 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:36.196013 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:36.197391 (+ 1378us) server_negotiation.cc:407] Connection header received
0811 19:11:36.199891 (+ 2500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:36.199897 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:36.199990 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:36.200090 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:36.201782 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:36.202604 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:36.204272 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:36.204698 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:36.204813 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:36.209405 (+ 4592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:36.209412 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:36.209415 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:36.209466 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:36.213354 (+ 3888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:36.214718 (+ 1364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:36.214725 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:36.214728 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:36.214959 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:36.215668 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:36.215673 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:36.215676 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:36.216140 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:36.216317 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:36.217308 (+ 991us) server_negotiation.cc:299] Negotiation successful
0811 19:11:36.217564 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":237,"threads_started":1}
I20260811 19:11:36.218855 11094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:36.196314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:37267 (local address 127.3.61.131:45249)
0811 19:11:36.197219 (+ 905us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:36.197270 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:36.199573 (+ 2303us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:36.200374 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:36.200384 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:36.200960 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:11:36.201588 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:36.201607 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:36.202782 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:36.202786 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:36.204109 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:36.204123 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:36.204961 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:36.204969 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:36.205217 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:36.205980 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:11:36.206006 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:11:36.208990 (+ 2984us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:11:36.213591 (+ 4601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:36.213605 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:36.213622 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:36.214382 (+ 760us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:36.215138 (+ 756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:36.215142 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:36.215145 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:36.215441 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:36.216343 (+ 902us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:11:36.216351 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:11:36.216772 (+ 421us) client_negotiation.cc:769] Sending connection context
0811 19:11:36.217050 (+ 278us) client_negotiation.cc:240] Negotiation successful
0811 19:11:36.217544 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":316,"threads_started":1}
I20260811 19:11:36.220743 11090 heartbeater.cc:347] Connected to a master server at 127.3.61.190:37267
I20260811 19:11:36.221084 11090 heartbeater.cc:464] Registering TS with master...
I20260811 19:11:36.221918 11090 heartbeater.cc:511] Master 127.3.61.190:37267 requested a full tablet report, sending...
I20260811 19:11:36.223626 10214 ts_manager.cc:194] Registered new tserver with Master: a68156b4a1cb44d2b66ea7511a4d304a (127.3.61.131:35265)
I20260811 19:11:36.224525 10214 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.131@KRBTEST.COM'} at 127.3.61.131:45249
I20260811 19:11:36.226361 3318 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 19:11:36 dist-test-slave-t7g5 krb5kdc[10141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475496, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 19:11:36 dist-test-slave-t7g5 krb5kdc[10141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475496, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.129@KRBTEST.COM
I20260811 19:11:36.295769 11103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:36.274691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37178 (local address 127.3.61.129:45141)
0811 19:11:36.275640 (+ 949us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:36.275647 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:36.275675 (+ 28us) server_negotiation.cc:407] Connection header received
0811 19:11:36.275740 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:36.275746 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:36.276093 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:36.276274 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:36.278558 (+ 2284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:36.279665 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:36.281452 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:36.282373 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:36.282550 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:36.288177 (+ 5627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:36.288202 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:36.288226 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:36.288285 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:36.291921 (+ 3636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:36.292883 (+ 962us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:36.292891 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:36.292894 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:36.293046 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:36.293539 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:36.293545 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:36.293549 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:36.294197 (+ 648us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:36.294533 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:36.294936 (+ 403us) server_negotiation.cc:299] Negotiation successful
0811 19:11:36.295284 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":740,"thread_start_us":513,"threads_started":1}
Aug 11 19:11:36 dist-test-slave-t7g5 krb5kdc[10141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475496, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:11:36.323257 11093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:36.302629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36592 (local address 127.3.61.190:37267)
0811 19:11:36.302837 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:36.302845 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:36.303079 (+ 234us) server_negotiation.cc:407] Connection header received
0811 19:11:36.303268 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:36.303274 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:36.303386 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:36.303489 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:36.305923 (+ 2434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:36.307049 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:36.309054 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:36.309749 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:36.309907 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:36.315995 (+ 6088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:36.316002 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:36.316007 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:36.316064 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:36.320334 (+ 4270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:36.321146 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:36.321152 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:36.321156 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:36.321292 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:36.321700 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:36.321705 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:36.321709 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:36.322120 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:36.322394 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:36.322698 (+ 304us) server_negotiation.cc:299] Negotiation successful
0811 19:11:36.322937 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
W20260811 19:11:36.323885 10214 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36592
I20260811 19:11:36.325004 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 10325
I20260811 19:11:36.343158 11103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:36.325999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37192 (local address 127.3.61.129:45141)
0811 19:11:36.326189 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:36.326195 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:36.326388 (+ 193us) server_negotiation.cc:407] Connection header received
0811 19:11:36.326607 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:36.326613 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:36.326718 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:36.326779 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:36.328170 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:36.328997 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:36.330536 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:36.331081 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:36.331170 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:36.334895 (+ 3725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:36.334906 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:36.334912 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:36.334987 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:36.339689 (+ 4702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:36.340787 (+ 1098us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:36.340796 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:36.340799 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:36.341066 (+ 267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:36.341487 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:36.341492 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:36.341496 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:36.341908 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:36.342178 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:36.342514 (+ 336us) server_negotiation.cc:299] Negotiation successful
0811 19:11:36.342797 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 19:11:36.344075 10567 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:11:36.434208 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 10325
I20260811 19:11:36.466878 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 10585
Aug 11 19:11:36 dist-test-slave-t7g5 krb5kdc[10141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475496, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.130@KRBTEST.COM
I20260811 19:11:36.487144 11111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:36.467904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47514 (local address 127.3.61.130:35237)
0811 19:11:36.468674 (+ 770us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:36.468684 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:36.468731 (+ 47us) server_negotiation.cc:407] Connection header received
0811 19:11:36.468835 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:36.468843 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:36.469165 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:36.469429 (+ 264us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:36.471263 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:36.472403 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:36.473954 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:36.474981 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:36.475195 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:36.480395 (+ 5200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:36.480415 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:36.480433 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:36.480495 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:36.483786 (+ 3291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:36.484629 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:36.484638 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:36.484640 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:36.484787 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:36.485179 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:36.485183 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:36.485185 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:36.485760 (+ 575us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:36.486127 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:36.486456 (+ 329us) server_negotiation.cc:299] Negotiation successful
0811 19:11:36.486775 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":512,"thread_start_us":250,"threads_started":1}
I20260811 19:11:36.488618 10828 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:11:36.578845 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 10585
I20260811 19:11:36.604301 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 10847
Aug 11 19:11:36 dist-test-slave-t7g5 krb5kdc[10141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475496, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.131@KRBTEST.COM
I20260811 19:11:36.622840 11115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:36.605565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52514 (local address 127.3.61.131:35265)
0811 19:11:36.606182 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:36.606189 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:36.606220 (+ 31us) server_negotiation.cc:407] Connection header received
0811 19:11:36.606283 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:36.606290 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:36.606514 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:36.606656 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:36.608010 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:36.609074 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:36.610434 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:36.611040 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:36.611161 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:36.616439 (+ 5278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:36.616456 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:36.616474 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:36.616523 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:36.619700 (+ 3177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:36.620524 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:36.620532 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:36.620534 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:36.620705 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:36.621093 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:36.621097 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:36.621099 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:36.621502 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:36.621849 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:36.622177 (+ 328us) server_negotiation.cc:299] Negotiation successful
0811 19:11:36.622489 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":308,"threads_started":1}
I20260811 19:11:36.624256 11078 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:11:36.713969 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 10847
I20260811 19:11:36.738227 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 10162
I20260811 19:11:36.739988 10266 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:11:36.853493 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 10162
2026-08-11T19:11:36Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4640 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:11:36 dist-test-slave-t7g5 krb5kdc[11128](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:11:36 dist-test-slave-t7g5 krb5kdc[11128](info): set up 2 sockets
Aug 11 19:11:36 dist-test-slave-t7g5 krb5kdc[11128](info): commencing operation
krb5kdc: starting...
W20260811 19:11:38.145522 3318 mini_kdc.cc:121] Time spent starting KDC: real 1.239s user 0.004s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Aug 11 19:11:38 dist-test-slave-t7g5 krb5kdc[11128](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475498, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:11:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:11:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.61.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab.
Entry for principal kudu/127.3.61.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab.
I20260811 19:11:38.446007 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:40163
--webserver_interface=127.3.61.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:33645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.61.190:40163
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:11:38.823156 11152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:11:38.823506 11152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:11:38.823622 11152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:11:38.831374 11152 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:11:38.831522 11152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:11:38.831588 11152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:11:38.831673 11152 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:11:38.831743 11152 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:11:38.842487 11152 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:33645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.61.190:40163
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:40163
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.61.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:11:38.844173 11152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:11:38.845677 11152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:11:38.860203 11167 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:38.860105 11160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:38.860116 11162 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:11:38.861706 11152 server_base.cc:1034] running on GCE node
I20260811 19:11:38.862679 11152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:11:38.864282 11152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:11:38.865586 11152 hybrid_clock.cc:648] HybridClock initialized: now 1786475498865547 us; error 41 us; skew 500 ppm
I20260811 19:11:38.866009 11152 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:11:38 dist-test-slave-t7g5 krb5kdc[11128](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475498, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:11:38.870169 11152 init.cc:377] Logged in from keytab as kudu/127.3.61.190@KRBTEST.COM (short username kudu)
I20260811 19:11:38.871654 11152 fs_manager.cc:359] Metadata directory not provided
I20260811 19:11:38.871764 11152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:11:38.872105 11152 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:11:38.875552 11152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/master-0/data/instance:
uuid: "a171d8157c744665970dcf334f8c8494"
format_stamp: "Formatted at 2026-08-11 19:11:38 on dist-test-slave-t7g5"
server_key: "e200822c154faa92e8c963c8f9995edf"
server_key_iv: "560faff6856dc791c24ec72bc806d24f"
server_key_version: "encryptionkey@0"
I20260811 19:11:38.876502 11152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/master-0/wal/instance:
uuid: "a171d8157c744665970dcf334f8c8494"
format_stamp: "Formatted at 2026-08-11 19:11:38 on dist-test-slave-t7g5"
server_key: "e200822c154faa92e8c963c8f9995edf"
server_key_iv: "560faff6856dc791c24ec72bc806d24f"
server_key_version: "encryptionkey@0"
I20260811 19:11:38.882611 11152 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.008s
I20260811 19:11:38.886556 11175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:38.888085 11152 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 19:11:38.888273 11152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/master-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/master-0/wal
uuid: "a171d8157c744665970dcf334f8c8494"
format_stamp: "Formatted at 2026-08-11 19:11:38 on dist-test-slave-t7g5"
server_key: "e200822c154faa92e8c963c8f9995edf"
server_key_iv: "560faff6856dc791c24ec72bc806d24f"
server_key_version: "encryptionkey@0"
I20260811 19:11:38.888515 11152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:11:38.932616 11152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:11:38.938419 11152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:11:38.938745 11152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:11:38.960968 11152 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.190:40163
I20260811 19:11:38.960916 11258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.190:40163 every 8 connection(s)
I20260811 19:11:38.962046 11152 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
I20260811 19:11:38.966028 11259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:11:38.968410 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 11152
I20260811 19:11:38.968645 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/master-0/wal/instance
I20260811 19:11:38.969311 3318 external_mini_cluster.cc:1428] Setting key c82aa8063f6580b8c2e349e2d3b374f5
I20260811 19:11:38.980085 11259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a171d8157c744665970dcf334f8c8494: Bootstrap starting.
I20260811 19:11:38.984685 11259 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a171d8157c744665970dcf334f8c8494: Neither blocks nor log segments found. Creating new log.
I20260811 19:11:38.985822 11259 log.cc:824] T 00000000000000000000000000000000 P a171d8157c744665970dcf334f8c8494: Log is configured to *not* fsync() on all Append() calls
Aug 11 19:11:38 dist-test-slave-t7g5 krb5kdc[11128](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475498, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:11:38.989027 11259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a171d8157c744665970dcf334f8c8494: No bootstrap required, opened a new log
I20260811 19:11:38.995934 11262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:38.971789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53122 (local address 127.3.61.190:40163)
0811 19:11:38.972869 (+ 1080us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:38.972885 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:38.973381 (+ 496us) server_negotiation.cc:407] Connection header received
0811 19:11:38.975460 (+ 2079us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:38.975512 (+ 52us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:38.976132 (+ 620us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:38.976659 (+ 527us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:38.978345 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:38.980323 (+ 1978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:38.981850 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:38.983012 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:38.983230 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:38.987638 (+ 4408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:38.987647 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:38.987664 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:38.987704 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:38.991022 (+ 3318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:38.992047 (+ 1025us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:38.992055 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:38.992066 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:38.992280 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:38.992784 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:38.992790 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:38.992793 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:38.993488 (+ 695us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:38.993833 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:38.994287 (+ 454us) server_negotiation.cc:299] Negotiation successful
0811 19:11:38.994723 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":782,"thread_start_us":494,"threads_started":1}
I20260811 19:11:38.998181 11259 raft_consensus.cc:348] T 00000000000000000000000000000000 P a171d8157c744665970dcf334f8c8494 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a171d8157c744665970dcf334f8c8494" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 40163 } }
I20260811 19:11:38.998584 11259 raft_consensus.cc:374] T 00000000000000000000000000000000 P a171d8157c744665970dcf334f8c8494 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:11:38.998682 11259 raft_consensus.cc:729] T 00000000000000000000000000000000 P a171d8157c744665970dcf334f8c8494 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a171d8157c744665970dcf334f8c8494, State: Initialized, Role: FOLLOWER
I20260811 19:11:38.999313 11259 consensus_queue.cc:260] T 00000000000000000000000000000000 P a171d8157c744665970dcf334f8c8494 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a171d8157c744665970dcf334f8c8494" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 40163 } }
I20260811 19:11:38.999543 11259 raft_consensus.cc:388] T 00000000000000000000000000000000 P a171d8157c744665970dcf334f8c8494 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:11:38.999641 11259 raft_consensus.cc:482] T 00000000000000000000000000000000 P a171d8157c744665970dcf334f8c8494 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:11:38.999775 11259 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a171d8157c744665970dcf334f8c8494 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:11:39.002097 11259 raft_consensus.cc:504] T 00000000000000000000000000000000 P a171d8157c744665970dcf334f8c8494 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a171d8157c744665970dcf334f8c8494" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 40163 } }
I20260811 19:11:39.002681 11259 leader_election.cc:302] T 00000000000000000000000000000000 P a171d8157c744665970dcf334f8c8494 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a171d8157c744665970dcf334f8c8494; no voters:
I20260811 19:11:39.003509 11259 leader_election.cc:288] T 00000000000000000000000000000000 P a171d8157c744665970dcf334f8c8494 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:11:39.003831 11267 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a171d8157c744665970dcf334f8c8494 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:11:39.004904 11267 raft_consensus.cc:686] T 00000000000000000000000000000000 P a171d8157c744665970dcf334f8c8494 [term 1 LEADER]: Becoming Leader. State: Replica: a171d8157c744665970dcf334f8c8494, State: Running, Role: LEADER
I20260811 19:11:39.005731 11267 consensus_queue.cc:237] T 00000000000000000000000000000000 P a171d8157c744665970dcf334f8c8494 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a171d8157c744665970dcf334f8c8494" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 40163 } }
I20260811 19:11:39.006317 11259 sys_catalog.cc:564] T 00000000000000000000000000000000 P a171d8157c744665970dcf334f8c8494 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:11:39.008690 11271 sys_catalog.cc:455] T 00000000000000000000000000000000 P a171d8157c744665970dcf334f8c8494 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a171d8157c744665970dcf334f8c8494. Latest consensus state: current_term: 1 leader_uuid: "a171d8157c744665970dcf334f8c8494" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a171d8157c744665970dcf334f8c8494" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 40163 } } }
I20260811 19:11:39.009070 11271 sys_catalog.cc:458] T 00000000000000000000000000000000 P a171d8157c744665970dcf334f8c8494 [sys.catalog]: This master's current role is: LEADER
I20260811 19:11:39.009469 11270 sys_catalog.cc:455] T 00000000000000000000000000000000 P a171d8157c744665970dcf334f8c8494 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a171d8157c744665970dcf334f8c8494" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a171d8157c744665970dcf334f8c8494" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 40163 } } }
I20260811 19:11:39.009828 11270 sys_catalog.cc:458] T 00000000000000000000000000000000 P a171d8157c744665970dcf334f8c8494 [sys.catalog]: This master's current role is: LEADER
I20260811 19:11:39.011293 11276 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:11:39.018711 11276 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:11:39.030391 11276 catalog_manager.cc:1347] Generated new cluster ID: d45ab400fc97487686aa20beb2ec601d
I20260811 19:11:39.030573 11276 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:11:39.058838 11276 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:11:39.061098 11276 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:11:39.078052 11276 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a171d8157c744665970dcf334f8c8494: Generated new TSK 0
I20260811 19:11:39.079362 11276 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.61.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab.
Entry for principal kudu/127.3.61.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab.
I20260811 19:11:39.146386 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.129:0
--local_ip_for_outbound_sockets=127.3.61.129
--webserver_interface=127.3.61.129
--webserver_port=0
--tserver_master_addrs=127.3.61.190:40163
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab
--principal=kudu/127.3.61.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:33645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:11:39.529796 11306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:11:39.530252 11306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:11:39.530396 11306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:11:39.539108 11306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:11:39.539258 11306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:11:39.539446 11306 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.129
I20260811 19:11:39.552791 11306 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:33645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab
--principal=kudu/127.3.61.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.61.129
--webserver_port=0
--tserver_master_addrs=127.3.61.190:40163
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.129
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:11:39.554421 11306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:11:39.555845 11306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:11:39.569643 11318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:39.569607 11315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:39.569620 11314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:11:39.570657 11306 server_base.cc:1034] running on GCE node
I20260811 19:11:39.571374 11306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:11:39.572476 11306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:11:39.573879 11306 hybrid_clock.cc:648] HybridClock initialized: now 1786475499573754 us; error 63 us; skew 500 ppm
I20260811 19:11:39.574448 11306 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:11:39 dist-test-slave-t7g5 krb5kdc[11128](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475499, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:11:39.578864 11306 init.cc:377] Logged in from keytab as kudu/127.3.61.129@KRBTEST.COM (short username kudu)
I20260811 19:11:39.580233 11306 fs_manager.cc:359] Metadata directory not provided
I20260811 19:11:39.580345 11306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:11:39.580631 11306 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:11:39.583909 11306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/ts-0/data/instance:
uuid: "54f6dd22a77b4c4e8ab990b649081ccc"
format_stamp: "Formatted at 2026-08-11 19:11:39 on dist-test-slave-t7g5"
server_key: "eb8c320ccac60c0e718a31b35bf845ab"
server_key_iv: "463cb4d104af4f80c38472e199825637"
server_key_version: "encryptionkey@0"
I20260811 19:11:39.584800 11306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/ts-0/wal/instance:
uuid: "54f6dd22a77b4c4e8ab990b649081ccc"
format_stamp: "Formatted at 2026-08-11 19:11:39 on dist-test-slave-t7g5"
server_key: "eb8c320ccac60c0e718a31b35bf845ab"
server_key_iv: "463cb4d104af4f80c38472e199825637"
server_key_version: "encryptionkey@0"
I20260811 19:11:39.591230 11306 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.002s
I20260811 19:11:39.595391 11325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:39.596947 11306 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260811 19:11:39.597137 11306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/ts-0/wal
uuid: "54f6dd22a77b4c4e8ab990b649081ccc"
format_stamp: "Formatted at 2026-08-11 19:11:39 on dist-test-slave-t7g5"
server_key: "eb8c320ccac60c0e718a31b35bf845ab"
server_key_iv: "463cb4d104af4f80c38472e199825637"
server_key_version: "encryptionkey@0"
I20260811 19:11:39.597376 11306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:11:39.622012 11306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:11:39.626948 11306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:11:39.627244 11306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:11:39.628356 11306 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:11:39.630055 11306 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:11:39.630163 11306 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:39.630271 11306 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:11:39.630331 11306 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:39.667776 11306 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.129:35399
I20260811 19:11:39.667824 11519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.129:35399 every 8 connection(s)
I20260811 19:11:39.668730 11306 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
I20260811 19:11:39.677172 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 11306
I20260811 19:11:39.677388 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/ts-0/wal/instance
I20260811 19:11:39.678108 3318 external_mini_cluster.cc:1428] Setting key c1a61826e0ec26245ba01b9971d26f81
Aug 11 19:11:39 dist-test-slave-t7g5 krb5kdc[11128](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475499, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.129@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:11:39.695267 11524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:39.671956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:48153 (local address 127.3.61.190:40163)
0811 19:11:39.672506 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:39.672510 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:39.673784 (+ 1274us) server_negotiation.cc:407] Connection header received
0811 19:11:39.676357 (+ 2573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:39.676364 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:39.676477 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:39.676591 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:39.679129 (+ 2538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:39.680036 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:39.681877 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:39.682466 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:39.682620 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:39.687299 (+ 4679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:39.687305 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:39.687309 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:39.687347 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:39.690736 (+ 3389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:39.691682 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:39.691688 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:39.691691 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:39.691856 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:39.692598 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:39.692606 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:39.692610 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:39.693251 (+ 641us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:39.693484 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:39.694581 (+ 1097us) server_negotiation.cc:299] Negotiation successful
0811 19:11:39.694895 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":297,"threads_started":1}
I20260811 19:11:39.696321 11525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:39.672541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:40163 (local address 127.3.61.129:48153)
0811 19:11:39.673478 (+ 937us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:39.673549 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:39.675954 (+ 2405us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:39.676941 (+ 987us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:39.676953 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:39.678034 (+ 1081us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:11:39.678855 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:39.678891 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:39.680251 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:39.680257 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:39.681698 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:39.681717 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:39.682762 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:39.682768 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:39.682995 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:39.683795 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:11:39.683824 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:11:39.686870 (+ 3046us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:11:39.690964 (+ 4094us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:39.690973 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:39.690989 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:39.691529 (+ 540us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:39.692022 (+ 493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:39.692026 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:39.692029 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:39.692282 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:39.693522 (+ 1240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:11:39.693532 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:11:39.694094 (+ 562us) client_negotiation.cc:769] Sending connection context
0811 19:11:39.694363 (+ 269us) client_negotiation.cc:240] Negotiation successful
0811 19:11:39.694868 (+ 505us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":359,"threads_started":1}
I20260811 19:11:39.698238 11520 heartbeater.cc:347] Connected to a master server at 127.3.61.190:40163
I20260811 19:11:39.698616 11520 heartbeater.cc:464] Registering TS with master...
I20260811 19:11:39.699438 11520 heartbeater.cc:511] Master 127.3.61.190:40163 requested a full tablet report, sending...
I20260811 19:11:39.702793 11206 ts_manager.cc:194] Registered new tserver with Master: 54f6dd22a77b4c4e8ab990b649081ccc (127.3.61.129:35399)
I20260811 19:11:39.704428 11206 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.129@KRBTEST.COM'} at 127.3.61.129:48153
WARNING: no policy specified for kudu/127.3.61.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab.
Entry for principal kudu/127.3.61.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab.
I20260811 19:11:39.741564 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.130:0
--local_ip_for_outbound_sockets=127.3.61.130
--webserver_interface=127.3.61.130
--webserver_port=0
--tserver_master_addrs=127.3.61.190:40163
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab
--principal=kudu/127.3.61.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:33645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:11:40.104737 11530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:11:40.105079 11530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:11:40.105250 11530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:11:40.113925 11530 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:11:40.114073 11530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:11:40.114346 11530 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.130
I20260811 19:11:40.124621 11530 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:33645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab
--principal=kudu/127.3.61.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.61.130
--webserver_port=0
--tserver_master_addrs=127.3.61.190:40163
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.130
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:11:40.126102 11530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:11:40.127525 11530 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:11:40.141428 11535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:40.141465 11536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:40.141638 11540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:11:40.142871 11530 server_base.cc:1034] running on GCE node
I20260811 19:11:40.143596 11530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:11:40.144672 11530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:11:40.145967 11530 hybrid_clock.cc:648] HybridClock initialized: now 1786475500145887 us; error 50 us; skew 500 ppm
I20260811 19:11:40.146503 11530 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:11:40 dist-test-slave-t7g5 krb5kdc[11128](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475500, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:11:40.155228 11530 init.cc:377] Logged in from keytab as kudu/127.3.61.130@KRBTEST.COM (short username kudu)
I20260811 19:11:40.156947 11530 fs_manager.cc:359] Metadata directory not provided
I20260811 19:11:40.157104 11530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:11:40.157471 11530 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:11:40.160799 11530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/ts-1/data/instance:
uuid: "ee5a41d7f2624bd991f00537c8543110"
format_stamp: "Formatted at 2026-08-11 19:11:40 on dist-test-slave-t7g5"
server_key: "fbebac529b29b1be973e917529e84f85"
server_key_iv: "2b6b5afd6f7c3b4a4207996362620a5a"
server_key_version: "encryptionkey@0"
I20260811 19:11:40.161739 11530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/ts-1/wal/instance:
uuid: "ee5a41d7f2624bd991f00537c8543110"
format_stamp: "Formatted at 2026-08-11 19:11:40 on dist-test-slave-t7g5"
server_key: "fbebac529b29b1be973e917529e84f85"
server_key_iv: "2b6b5afd6f7c3b4a4207996362620a5a"
server_key_version: "encryptionkey@0"
I20260811 19:11:40.167850 11530 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260811 19:11:40.173071 11558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:40.175125 11530 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260811 19:11:40.175380 11530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/ts-1/wal
uuid: "ee5a41d7f2624bd991f00537c8543110"
format_stamp: "Formatted at 2026-08-11 19:11:40 on dist-test-slave-t7g5"
server_key: "fbebac529b29b1be973e917529e84f85"
server_key_iv: "2b6b5afd6f7c3b4a4207996362620a5a"
server_key_version: "encryptionkey@0"
I20260811 19:11:40.175660 11530 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:11:40.199103 11530 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:11:40.205315 11530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:11:40.205726 11530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:11:40.206903 11530 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:11:40.208449 11530 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:11:40.208546 11530 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:40.208654 11530 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:11:40.208717 11530 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:40.244896 11530 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.130:35161
I20260811 19:11:40.244939 11756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.130:35161 every 8 connection(s)
I20260811 19:11:40.245920 11530 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
I20260811 19:11:40.253288 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 11530
I20260811 19:11:40.253480 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/ts-1/wal/instance
I20260811 19:11:40.253960 3318 external_mini_cluster.cc:1428] Setting key d1c18678b1039b94bd14bb5f03c265af
Aug 11 19:11:40 dist-test-slave-t7g5 krb5kdc[11128](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475500, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.130@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:11:40.271908 11762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:40.248791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:40399 (local address 127.3.61.190:40163)
0811 19:11:40.249454 (+ 663us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:40.249459 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:40.250680 (+ 1221us) server_negotiation.cc:407] Connection header received
0811 19:11:40.252584 (+ 1904us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:40.252591 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:40.252686 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:40.252792 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:40.255335 (+ 2543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:40.256247 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:40.258526 (+ 2279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:40.259277 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:40.259452 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:40.264692 (+ 5240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:40.264699 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:40.264702 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:40.264744 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:40.267691 (+ 2947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:40.268925 (+ 1234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:40.268931 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:40.268934 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:40.269102 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:40.269898 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:40.269903 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:40.269907 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:40.270292 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:40.270442 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:40.271298 (+ 856us) server_negotiation.cc:299] Negotiation successful
0811 19:11:40.271546 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":395,"threads_started":1}
I20260811 19:11:40.273140 11764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:40.249362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:40163 (local address 127.3.61.130:40399)
0811 19:11:40.250458 (+ 1096us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:40.250516 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:40.252270 (+ 1754us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:40.253721 (+ 1451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:40.253730 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:40.254363 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:11:40.255118 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:40.255140 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:40.256547 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:40.256555 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:40.258290 (+ 1735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:40.258315 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:40.259657 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:40.259664 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:40.259941 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:40.261059 (+ 1118us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:11:40.261098 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:11:40.264312 (+ 3214us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:11:40.267953 (+ 3641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:40.267967 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:40.267989 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:40.268711 (+ 722us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:40.269288 (+ 577us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:40.269292 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:40.269296 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:40.269740 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:40.270482 (+ 742us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:11:40.270491 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:11:40.270896 (+ 405us) client_negotiation.cc:769] Sending connection context
0811 19:11:40.271160 (+ 264us) client_negotiation.cc:240] Negotiation successful
0811 19:11:40.271934 (+ 774us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":784,"thread_start_us":410,"threads_started":1}
I20260811 19:11:40.274914 11757 heartbeater.cc:347] Connected to a master server at 127.3.61.190:40163
I20260811 19:11:40.275269 11757 heartbeater.cc:464] Registering TS with master...
I20260811 19:11:40.276196 11757 heartbeater.cc:511] Master 127.3.61.190:40163 requested a full tablet report, sending...
I20260811 19:11:40.278270 11206 ts_manager.cc:194] Registered new tserver with Master: ee5a41d7f2624bd991f00537c8543110 (127.3.61.130:35161)
I20260811 19:11:40.279158 11206 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.130@KRBTEST.COM'} at 127.3.61.130:40399
WARNING: no policy specified for kudu/127.3.61.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab.
Entry for principal kudu/127.3.61.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab.
I20260811 19:11:40.316862 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.131:0
--local_ip_for_outbound_sockets=127.3.61.131
--webserver_interface=127.3.61.131
--webserver_port=0
--tserver_master_addrs=127.3.61.190:40163
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab
--principal=kudu/127.3.61.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:33645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:11:40.689592 11768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:11:40.689953 11768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:11:40.690102 11768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:11:40.699168 11768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:11:40.699318 11768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:11:40.699456 11768 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.131
I20260811 19:11:40.708309 11520 heartbeater.cc:503] Master 127.3.61.190:40163 was elected leader, sending a full tablet report...
I20260811 19:11:40.710752 11768 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:33645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab
--principal=kudu/127.3.61.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.61.131
--webserver_port=0
--tserver_master_addrs=127.3.61.190:40163
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.131
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:11:40.712121 11768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:11:40.713723 11768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:11:40.728744 11783 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:40.728950 11776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:40.728856 11777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:11:40.729972 11768 server_base.cc:1034] running on GCE node
I20260811 19:11:40.730680 11768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:11:40.731745 11768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:11:40.733016 11768 hybrid_clock.cc:648] HybridClock initialized: now 1786475500732947 us; error 54 us; skew 500 ppm
I20260811 19:11:40.733397 11768 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:11:40 dist-test-slave-t7g5 krb5kdc[11128](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475500, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:11:40.736840 11768 init.cc:377] Logged in from keytab as kudu/127.3.61.131@KRBTEST.COM (short username kudu)
I20260811 19:11:40.739270 11768 fs_manager.cc:359] Metadata directory not provided
I20260811 19:11:40.739424 11768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:11:40.739957 11768 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:11:40.743887 11768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/ts-2/data/instance:
uuid: "6a611d27b0c54f40be9fa0a8ac450719"
format_stamp: "Formatted at 2026-08-11 19:11:40 on dist-test-slave-t7g5"
server_key: "381f7fda0d7e37f8d642e7240fe27822"
server_key_iv: "f2d58872dab47e2283a165b3556ae99f"
server_key_version: "encryptionkey@0"
I20260811 19:11:40.745023 11768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/ts-2/wal/instance:
uuid: "6a611d27b0c54f40be9fa0a8ac450719"
format_stamp: "Formatted at 2026-08-11 19:11:40 on dist-test-slave-t7g5"
server_key: "381f7fda0d7e37f8d642e7240fe27822"
server_key_iv: "f2d58872dab47e2283a165b3556ae99f"
server_key_version: "encryptionkey@0"
I20260811 19:11:40.751552 11768 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260811 19:11:40.755815 11798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:40.758126 11768 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 19:11:40.758399 11768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/ts-2/wal
uuid: "6a611d27b0c54f40be9fa0a8ac450719"
format_stamp: "Formatted at 2026-08-11 19:11:40 on dist-test-slave-t7g5"
server_key: "381f7fda0d7e37f8d642e7240fe27822"
server_key_iv: "f2d58872dab47e2283a165b3556ae99f"
server_key_version: "encryptionkey@0"
I20260811 19:11:40.758698 11768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:11:40.782116 11768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:11:40.786865 11768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:11:40.787137 11768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:11:40.788257 11768 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:11:40.789994 11768 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:11:40.790102 11768 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:40.790195 11768 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:11:40.790253 11768 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:40.829241 11768 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.131:35911
I20260811 19:11:40.829286 12000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.131:35911 every 8 connection(s)
I20260811 19:11:40.830168 11768 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
I20260811 19:11:40.835630 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 11768
I20260811 19:11:40.835836 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/ts-2/wal/instance
I20260811 19:11:40.836293 3318 external_mini_cluster.cc:1428] Setting key 123555f027541dd2fc68cd0e25c85208
Aug 11 19:11:40 dist-test-slave-t7g5 krb5kdc[11128](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475500, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.131@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:11:40.853937 12006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:40.832864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:40015 (local address 127.3.61.190:40163)
0811 19:11:40.833381 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:40.833388 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:40.834364 (+ 976us) server_negotiation.cc:407] Connection header received
0811 19:11:40.836316 (+ 1952us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:40.836322 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:40.836422 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:40.836567 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:40.838962 (+ 2395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:40.839905 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:40.841559 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:40.842054 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:40.842166 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:40.846160 (+ 3994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:40.846165 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:40.846168 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:40.846205 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:40.849493 (+ 3288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:40.850427 (+ 934us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:40.850434 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:40.850437 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:40.850612 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:40.851322 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:40.851330 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:40.851334 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:40.851900 (+ 566us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:40.852092 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:40.853137 (+ 1045us) server_negotiation.cc:299] Negotiation successful
0811 19:11:40.853471 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":228,"threads_started":1}
I20260811 19:11:40.855276 12007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:40.833400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:40163 (local address 127.3.61.131:40015)
0811 19:11:40.834191 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:40.834238 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:40.836015 (+ 1777us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:40.837200 (+ 1185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:40.837215 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:40.837943 (+ 728us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:11:40.838667 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:40.838695 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:40.840067 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:40.840074 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:40.841393 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:40.841407 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:40.842352 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:40.842358 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:40.842566 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:40.843263 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:11:40.843287 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:11:40.845928 (+ 2641us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:11:40.849723 (+ 3795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:40.849732 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:40.849747 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:40.850260 (+ 513us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:40.850773 (+ 513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:40.850777 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:40.850779 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:40.851093 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:40.852088 (+ 995us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:11:40.852095 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:11:40.852557 (+ 462us) client_negotiation.cc:769] Sending connection context
0811 19:11:40.852893 (+ 336us) client_negotiation.cc:240] Negotiation successful
0811 19:11:40.853487 (+ 594us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":339,"threads_started":1}
I20260811 19:11:40.857146 12001 heartbeater.cc:347] Connected to a master server at 127.3.61.190:40163
I20260811 19:11:40.857508 12001 heartbeater.cc:464] Registering TS with master...
I20260811 19:11:40.858237 12001 heartbeater.cc:511] Master 127.3.61.190:40163 requested a full tablet report, sending...
I20260811 19:11:40.859829 11206 ts_manager.cc:194] Registered new tserver with Master: 6a611d27b0c54f40be9fa0a8ac450719 (127.3.61.131:35911)
I20260811 19:11:40.860675 11206 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.131@KRBTEST.COM'} at 127.3.61.131:40015
I20260811 19:11:40.866276 3318 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 19:11:40.886906 12006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:40.871494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53136 (local address 127.3.61.190:40163)
0811 19:11:40.871830 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:40.871838 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:40.872244 (+ 406us) server_negotiation.cc:407] Connection header received
0811 19:11:40.872530 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:40.872553 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:40.872677 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:40.872829 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:40.874751 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:40.875812 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:40.877500 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:40.878098 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:40.878206 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:40.880608 (+ 2402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:40.880613 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:40.880616 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:40.880651 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:40.883622 (+ 2971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:40.884296 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:40.884302 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:40.884306 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:40.884560 (+ 254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:40.885086 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:40.885092 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:40.885095 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:40.885456 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:40.885832 (+ 376us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:40.886264 (+ 432us) server_negotiation.cc:299] Negotiation successful
0811 19:11:40.886531 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
I20260811 19:11:40.892203 11206 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53136:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 19:11:40.899755 11206 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:11:40.941027 12033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:40.915314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.131:35911 (local address 127.0.0.1:49000)
0811 19:11:40.928321 (+ 13007us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:40.928362 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:40.928791 (+ 429us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:40.930920 (+ 2129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:40.930930 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:40.930968 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:11:40.931709 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:40.931743 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:40.934872 (+ 3129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:40.934880 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:40.937732 (+ 2852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:40.937750 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:40.940067 (+ 2317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:40.940074 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:40.940271 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:40.940316 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 19:11:40.940431 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 19:11:40.940648 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12688,"thread_start_us":283,"threads_started":1}
I20260811 19:11:40.941031 12032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:40.915569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49000 (local address 127.3.61.131:35911)
0811 19:11:40.929782 (+ 14213us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:40.929792 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:40.929834 (+ 42us) server_negotiation.cc:407] Connection header received
0811 19:11:40.929964 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:40.929974 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:40.930290 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:40.930557 (+ 267us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:11:40.932155 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:40.934603 (+ 2448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:40.937982 (+ 3379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:40.939518 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:40.939646 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:40.939916 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:40.940518 (+ 602us) server_negotiation.cc:299] Negotiation successful
0811 19:11:40.940688 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14004,"thread_start_us":13800,"threads_started":1}
I20260811 19:11:40.944883 11880 tablet_service.cc:1467] Processing CreateTablet for tablet 3b155fd608be48ceab75173c660a66c2 (DEFAULT_TABLE table=test-table [id=dcc3fca4fc2a4cb996ecca812440f5bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:11:40.946394 11880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b155fd608be48ceab75173c660a66c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:11:40.947814 12026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:40.914959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:35399 (local address 127.0.0.1:34214)
0811 19:11:40.916132 (+ 1173us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:40.916181 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:40.916527 (+ 346us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:40.938965 (+ 22438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:40.938972 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:40.939013 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:11:40.939530 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:40.939547 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:40.941818 (+ 2271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:40.941824 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:40.944660 (+ 2836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:40.944678 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:40.946969 (+ 2291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:40.946974 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:40.947131 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:40.947160 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 19:11:40.947257 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 19:11:40.947413 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":842,"thread_start_us":22468,"threads_started":1}
I20260811 19:11:40.947834 12030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:40.915333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34214 (local address 127.3.61.129:35399)
0811 19:11:40.938204 (+ 22871us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:40.938215 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:40.938251 (+ 36us) server_negotiation.cc:407] Connection header received
0811 19:11:40.938385 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:40.938394 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:40.938695 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:40.938950 (+ 255us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:11:40.939824 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:40.941596 (+ 1772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:40.944920 (+ 3324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:40.946468 (+ 1548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:40.946616 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:40.946810 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:40.947342 (+ 532us) server_negotiation.cc:299] Negotiation successful
0811 19:11:40.947526 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22638,"thread_start_us":22785,"threads_started":1}
I20260811 19:11:40.951553 12025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:40.917067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.130:35161 (local address 127.0.0.1:41866)
0811 19:11:40.937190 (+ 20123us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:40.937210 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:40.937349 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:40.939626 (+ 2277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:40.939631 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:40.939665 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:11:40.940098 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:40.940111 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:40.946335 (+ 6224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:40.946342 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:40.948564 (+ 2222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:40.948581 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:40.950922 (+ 2341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:40.950927 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:40.951057 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:40.951082 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 19:11:40.951150 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 19:11:40.951269 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19969,"mutex_wait_us":1,"thread_start_us":11887,"threads_started":1}
I20260811 19:11:40.951756 12034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:40.917959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41866 (local address 127.3.61.130:35161)
0811 19:11:40.938785 (+ 20826us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:40.938793 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:40.938824 (+ 31us) server_negotiation.cc:407] Connection header received
0811 19:11:40.938939 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:40.938948 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:40.939329 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:40.939614 (+ 285us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:11:40.940294 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:40.946121 (+ 5827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:40.948826 (+ 2705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:40.950600 (+ 1774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:40.950887 (+ 287us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:40.951108 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:40.951206 (+ 98us) server_negotiation.cc:299] Negotiation successful
0811 19:11:40.951397 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20636,"thread_start_us":21198,"threads_started":1}
I20260811 19:11:40.952608 11407 tablet_service.cc:1467] Processing CreateTablet for tablet 3b155fd608be48ceab75173c660a66c2 (DEFAULT_TABLE table=test-table [id=dcc3fca4fc2a4cb996ecca812440f5bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:11:40.954061 11407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b155fd608be48ceab75173c660a66c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:11:40.955649 11639 tablet_service.cc:1467] Processing CreateTablet for tablet 3b155fd608be48ceab75173c660a66c2 (DEFAULT_TABLE table=test-table [id=dcc3fca4fc2a4cb996ecca812440f5bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:11:40.957144 11639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b155fd608be48ceab75173c660a66c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:11:40.970739 12039 tablet_bootstrap.cc:492] T 3b155fd608be48ceab75173c660a66c2 P 54f6dd22a77b4c4e8ab990b649081ccc: Bootstrap starting.
I20260811 19:11:40.972184 12036 tablet_bootstrap.cc:492] T 3b155fd608be48ceab75173c660a66c2 P 6a611d27b0c54f40be9fa0a8ac450719: Bootstrap starting.
I20260811 19:11:40.974609 12039 tablet_bootstrap.cc:654] T 3b155fd608be48ceab75173c660a66c2 P 54f6dd22a77b4c4e8ab990b649081ccc: Neither blocks nor log segments found. Creating new log.
I20260811 19:11:40.975925 12036 tablet_bootstrap.cc:654] T 3b155fd608be48ceab75173c660a66c2 P 6a611d27b0c54f40be9fa0a8ac450719: Neither blocks nor log segments found. Creating new log.
I20260811 19:11:40.976075 12039 log.cc:824] T 3b155fd608be48ceab75173c660a66c2 P 54f6dd22a77b4c4e8ab990b649081ccc: Log is configured to *not* fsync() on all Append() calls
I20260811 19:11:40.977205 12036 log.cc:824] T 3b155fd608be48ceab75173c660a66c2 P 6a611d27b0c54f40be9fa0a8ac450719: Log is configured to *not* fsync() on all Append() calls
I20260811 19:11:40.981233 12046 tablet_bootstrap.cc:492] T 3b155fd608be48ceab75173c660a66c2 P ee5a41d7f2624bd991f00537c8543110: Bootstrap starting.
I20260811 19:11:40.985452 12046 tablet_bootstrap.cc:654] T 3b155fd608be48ceab75173c660a66c2 P ee5a41d7f2624bd991f00537c8543110: Neither blocks nor log segments found. Creating new log.
I20260811 19:11:40.986908 12046 log.cc:824] T 3b155fd608be48ceab75173c660a66c2 P ee5a41d7f2624bd991f00537c8543110: Log is configured to *not* fsync() on all Append() calls
I20260811 19:11:40.989044 12039 tablet_bootstrap.cc:492] T 3b155fd608be48ceab75173c660a66c2 P 54f6dd22a77b4c4e8ab990b649081ccc: No bootstrap required, opened a new log
I20260811 19:11:40.989185 12036 tablet_bootstrap.cc:492] T 3b155fd608be48ceab75173c660a66c2 P 6a611d27b0c54f40be9fa0a8ac450719: No bootstrap required, opened a new log
I20260811 19:11:40.989375 12039 ts_tablet_manager.cc:1397] T 3b155fd608be48ceab75173c660a66c2 P 54f6dd22a77b4c4e8ab990b649081ccc: Time spent bootstrapping tablet: real 0.019s user 0.011s sys 0.006s
I20260811 19:11:40.989511 12036 ts_tablet_manager.cc:1397] T 3b155fd608be48ceab75173c660a66c2 P 6a611d27b0c54f40be9fa0a8ac450719: Time spent bootstrapping tablet: real 0.018s user 0.000s sys 0.006s
I20260811 19:11:40.991935 12046 tablet_bootstrap.cc:492] T 3b155fd608be48ceab75173c660a66c2 P ee5a41d7f2624bd991f00537c8543110: No bootstrap required, opened a new log
I20260811 19:11:40.992234 12046 ts_tablet_manager.cc:1397] T 3b155fd608be48ceab75173c660a66c2 P ee5a41d7f2624bd991f00537c8543110: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.005s
I20260811 19:11:40.997386 12039 raft_consensus.cc:348] T 3b155fd608be48ceab75173c660a66c2 P 54f6dd22a77b4c4e8ab990b649081ccc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54f6dd22a77b4c4e8ab990b649081ccc" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 35399 } } peers { permanent_uuid: "6a611d27b0c54f40be9fa0a8ac450719" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 35911 } } peers { permanent_uuid: "ee5a41d7f2624bd991f00537c8543110" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 35161 } }
I20260811 19:11:40.997838 12039 raft_consensus.cc:374] T 3b155fd608be48ceab75173c660a66c2 P 54f6dd22a77b4c4e8ab990b649081ccc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:11:40.997944 12039 raft_consensus.cc:729] T 3b155fd608be48ceab75173c660a66c2 P 54f6dd22a77b4c4e8ab990b649081ccc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54f6dd22a77b4c4e8ab990b649081ccc, State: Initialized, Role: FOLLOWER
I20260811 19:11:40.998558 12039 consensus_queue.cc:260] T 3b155fd608be48ceab75173c660a66c2 P 54f6dd22a77b4c4e8ab990b649081ccc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54f6dd22a77b4c4e8ab990b649081ccc" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 35399 } } peers { permanent_uuid: "6a611d27b0c54f40be9fa0a8ac450719" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 35911 } } peers { permanent_uuid: "ee5a41d7f2624bd991f00537c8543110" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 35161 } }
I20260811 19:11:41.000072 12036 raft_consensus.cc:348] T 3b155fd608be48ceab75173c660a66c2 P 6a611d27b0c54f40be9fa0a8ac450719 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54f6dd22a77b4c4e8ab990b649081ccc" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 35399 } } peers { permanent_uuid: "6a611d27b0c54f40be9fa0a8ac450719" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 35911 } } peers { permanent_uuid: "ee5a41d7f2624bd991f00537c8543110" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 35161 } }
I20260811 19:11:41.000547 12036 raft_consensus.cc:374] T 3b155fd608be48ceab75173c660a66c2 P 6a611d27b0c54f40be9fa0a8ac450719 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:11:41.000644 12036 raft_consensus.cc:729] T 3b155fd608be48ceab75173c660a66c2 P 6a611d27b0c54f40be9fa0a8ac450719 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a611d27b0c54f40be9fa0a8ac450719, State: Initialized, Role: FOLLOWER
I20260811 19:11:41.001281 12036 consensus_queue.cc:260] T 3b155fd608be48ceab75173c660a66c2 P 6a611d27b0c54f40be9fa0a8ac450719 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54f6dd22a77b4c4e8ab990b649081ccc" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 35399 } } peers { permanent_uuid: "6a611d27b0c54f40be9fa0a8ac450719" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 35911 } } peers { permanent_uuid: "ee5a41d7f2624bd991f00537c8543110" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 35161 } }
I20260811 19:11:41.002413 12039 ts_tablet_manager.cc:1428] T 3b155fd608be48ceab75173c660a66c2 P 54f6dd22a77b4c4e8ab990b649081ccc: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260811 19:11:41.003624 12001 heartbeater.cc:503] Master 127.3.61.190:40163 was elected leader, sending a full tablet report...
I20260811 19:11:41.003376 12046 raft_consensus.cc:348] T 3b155fd608be48ceab75173c660a66c2 P ee5a41d7f2624bd991f00537c8543110 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54f6dd22a77b4c4e8ab990b649081ccc" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 35399 } } peers { permanent_uuid: "6a611d27b0c54f40be9fa0a8ac450719" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 35911 } } peers { permanent_uuid: "ee5a41d7f2624bd991f00537c8543110" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 35161 } }
I20260811 19:11:41.003863 12046 raft_consensus.cc:374] T 3b155fd608be48ceab75173c660a66c2 P ee5a41d7f2624bd991f00537c8543110 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:11:41.003973 12046 raft_consensus.cc:729] T 3b155fd608be48ceab75173c660a66c2 P ee5a41d7f2624bd991f00537c8543110 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5a41d7f2624bd991f00537c8543110, State: Initialized, Role: FOLLOWER
I20260811 19:11:41.004339 12036 ts_tablet_manager.cc:1428] T 3b155fd608be48ceab75173c660a66c2 P 6a611d27b0c54f40be9fa0a8ac450719: Time spent starting tablet: real 0.015s user 0.013s sys 0.003s
I20260811 19:11:41.004792 12046 consensus_queue.cc:260] T 3b155fd608be48ceab75173c660a66c2 P ee5a41d7f2624bd991f00537c8543110 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54f6dd22a77b4c4e8ab990b649081ccc" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 35399 } } peers { permanent_uuid: "6a611d27b0c54f40be9fa0a8ac450719" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 35911 } } peers { permanent_uuid: "ee5a41d7f2624bd991f00537c8543110" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 35161 } }
I20260811 19:11:41.006009 11757 heartbeater.cc:503] Master 127.3.61.190:40163 was elected leader, sending a full tablet report...
I20260811 19:11:41.007020 12046 ts_tablet_manager.cc:1428] T 3b155fd608be48ceab75173c660a66c2 P ee5a41d7f2624bd991f00537c8543110: Time spent starting tablet: real 0.015s user 0.006s sys 0.010s
I20260811 19:11:41.041354 12054 raft_consensus.cc:482] T 3b155fd608be48ceab75173c660a66c2 P ee5a41d7f2624bd991f00537c8543110 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:11:41.041847 12054 raft_consensus.cc:504] T 3b155fd608be48ceab75173c660a66c2 P ee5a41d7f2624bd991f00537c8543110 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54f6dd22a77b4c4e8ab990b649081ccc" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 35399 } } peers { permanent_uuid: "6a611d27b0c54f40be9fa0a8ac450719" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 35911 } } peers { permanent_uuid: "ee5a41d7f2624bd991f00537c8543110" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 35161 } }
I20260811 19:11:41.044165 12054 leader_election.cc:288] T 3b155fd608be48ceab75173c660a66c2 P ee5a41d7f2624bd991f00537c8543110 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 54f6dd22a77b4c4e8ab990b649081ccc (127.3.61.129:35399), 6a611d27b0c54f40be9fa0a8ac450719 (127.3.61.131:35911)
I20260811 19:11:41.052788 12032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:41.044886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:38231 (local address 127.3.61.131:35911)
0811 19:11:41.046175 (+ 1289us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:41.046183 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:41.047025 (+ 842us) server_negotiation.cc:407] Connection header received
0811 19:11:41.047190 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:41.047194 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:41.047257 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:41.047339 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:11:41.048082 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:41.048854 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:41.051009 (+ 2155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:41.051828 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:41.052013 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:41.052089 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:41.052394 (+ 305us) server_negotiation.cc:299] Negotiation successful
0811 19:11:41.052544 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":665}
I20260811 19:11:41.052794 12030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:41.045978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:32807 (local address 127.3.61.129:35399)
0811 19:11:41.046366 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:41.046373 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:41.046979 (+ 606us) server_negotiation.cc:407] Connection header received
0811 19:11:41.047175 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:41.047180 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:41.047245 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:41.047324 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:11:41.048235 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:41.049084 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:41.051006 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:41.051768 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:41.051919 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:41.052000 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:41.052421 (+ 421us) server_negotiation.cc:299] Negotiation successful
0811 19:11:41.052555 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179}
I20260811 19:11:41.053062 12058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:41.044721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:35399 (local address 127.3.61.130:32807)
0811 19:11:41.046854 (+ 2133us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:41.046871 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:41.047024 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:41.047381 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:41.047385 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:41.047434 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:11:41.048054 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:41.048071 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:41.049233 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:41.049237 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:41.050834 (+ 1597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:41.050848 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:41.052018 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:41.052025 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:41.052252 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:41.052276 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 19:11:41.052367 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 19:11:41.052513 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1981,"mutex_wait_us":39,"spinlock_wait_cycles":5376,"thread_start_us":1770,"threads_started":1}
I20260811 19:11:41.053381 12057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:41.044872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.131:35911 (local address 127.3.61.130:38231)
0811 19:11:41.046899 (+ 2027us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:41.046915 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:41.047060 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:41.047380 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:41.047385 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:41.047425 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:11:41.047858 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:41.047869 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:41.049013 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:41.049019 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:41.050824 (+ 1805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:41.050840 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:41.052070 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:41.052075 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:41.052227 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:41.052252 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 19:11:41.052337 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 19:11:41.052479 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1919,"mutex_wait_us":107,"spinlock_wait_cycles":2304,"thread_start_us":1513,"threads_started":1}
I20260811 19:11:41.054219 11435 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b155fd608be48ceab75173c660a66c2" candidate_uuid: "ee5a41d7f2624bd991f00537c8543110" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54f6dd22a77b4c4e8ab990b649081ccc" is_pre_election: true
I20260811 19:11:41.054653 11921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b155fd608be48ceab75173c660a66c2" candidate_uuid: "ee5a41d7f2624bd991f00537c8543110" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a611d27b0c54f40be9fa0a8ac450719" is_pre_election: true
I20260811 19:11:41.054776 11435 raft_consensus.cc:2454] T 3b155fd608be48ceab75173c660a66c2 P 54f6dd22a77b4c4e8ab990b649081ccc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee5a41d7f2624bd991f00537c8543110 in term 0.
I20260811 19:11:41.055142 11921 raft_consensus.cc:2454] T 3b155fd608be48ceab75173c660a66c2 P 6a611d27b0c54f40be9fa0a8ac450719 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee5a41d7f2624bd991f00537c8543110 in term 0.
I20260811 19:11:41.055548 11559 leader_election.cc:302] T 3b155fd608be48ceab75173c660a66c2 P ee5a41d7f2624bd991f00537c8543110 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54f6dd22a77b4c4e8ab990b649081ccc, ee5a41d7f2624bd991f00537c8543110; no voters:
I20260811 19:11:41.056005 12054 raft_consensus.cc:2781] T 3b155fd608be48ceab75173c660a66c2 P ee5a41d7f2624bd991f00537c8543110 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:11:41.056173 12054 raft_consensus.cc:482] T 3b155fd608be48ceab75173c660a66c2 P ee5a41d7f2624bd991f00537c8543110 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:11:41.056286 12054 raft_consensus.cc:3037] T 3b155fd608be48ceab75173c660a66c2 P ee5a41d7f2624bd991f00537c8543110 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:11:41.058699 12054 raft_consensus.cc:504] T 3b155fd608be48ceab75173c660a66c2 P ee5a41d7f2624bd991f00537c8543110 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54f6dd22a77b4c4e8ab990b649081ccc" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 35399 } } peers { permanent_uuid: "6a611d27b0c54f40be9fa0a8ac450719" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 35911 } } peers { permanent_uuid: "ee5a41d7f2624bd991f00537c8543110" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 35161 } }
I20260811 19:11:41.059371 12054 leader_election.cc:288] T 3b155fd608be48ceab75173c660a66c2 P ee5a41d7f2624bd991f00537c8543110 [CANDIDATE]: Term 1 election: Requested vote from peers 54f6dd22a77b4c4e8ab990b649081ccc (127.3.61.129:35399), 6a611d27b0c54f40be9fa0a8ac450719 (127.3.61.131:35911)
I20260811 19:11:41.060001 11435 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b155fd608be48ceab75173c660a66c2" candidate_uuid: "ee5a41d7f2624bd991f00537c8543110" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54f6dd22a77b4c4e8ab990b649081ccc"
I20260811 19:11:41.060016 11921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b155fd608be48ceab75173c660a66c2" candidate_uuid: "ee5a41d7f2624bd991f00537c8543110" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a611d27b0c54f40be9fa0a8ac450719"
I20260811 19:11:41.060226 11435 raft_consensus.cc:3037] T 3b155fd608be48ceab75173c660a66c2 P 54f6dd22a77b4c4e8ab990b649081ccc [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:11:41.060212 11921 raft_consensus.cc:3037] T 3b155fd608be48ceab75173c660a66c2 P 6a611d27b0c54f40be9fa0a8ac450719 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:11:41.062568 11435 raft_consensus.cc:2454] T 3b155fd608be48ceab75173c660a66c2 P 54f6dd22a77b4c4e8ab990b649081ccc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5a41d7f2624bd991f00537c8543110 in term 1.
I20260811 19:11:41.062639 11921 raft_consensus.cc:2454] T 3b155fd608be48ceab75173c660a66c2 P 6a611d27b0c54f40be9fa0a8ac450719 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5a41d7f2624bd991f00537c8543110 in term 1.
I20260811 19:11:41.063031 11559 leader_election.cc:302] T 3b155fd608be48ceab75173c660a66c2 P ee5a41d7f2624bd991f00537c8543110 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54f6dd22a77b4c4e8ab990b649081ccc, ee5a41d7f2624bd991f00537c8543110; no voters:
I20260811 19:11:41.063351 12054 raft_consensus.cc:2781] T 3b155fd608be48ceab75173c660a66c2 P ee5a41d7f2624bd991f00537c8543110 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:11:41.063954 12054 raft_consensus.cc:686] T 3b155fd608be48ceab75173c660a66c2 P ee5a41d7f2624bd991f00537c8543110 [term 1 LEADER]: Becoming Leader. State: Replica: ee5a41d7f2624bd991f00537c8543110, State: Running, Role: LEADER
I20260811 19:11:41.064916 12054 consensus_queue.cc:237] T 3b155fd608be48ceab75173c660a66c2 P ee5a41d7f2624bd991f00537c8543110 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54f6dd22a77b4c4e8ab990b649081ccc" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 35399 } } peers { permanent_uuid: "6a611d27b0c54f40be9fa0a8ac450719" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 35911 } } peers { permanent_uuid: "ee5a41d7f2624bd991f00537c8543110" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 35161 } }
I20260811 19:11:41.069963 11203 catalog_manager.cc:5495] T 3b155fd608be48ceab75173c660a66c2 P ee5a41d7f2624bd991f00537c8543110 reported cstate change: term changed from 0 to 1, leader changed from <none> to ee5a41d7f2624bd991f00537c8543110 (127.3.61.130). New cstate: current_term: 1 leader_uuid: "ee5a41d7f2624bd991f00537c8543110" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54f6dd22a77b4c4e8ab990b649081ccc" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 35399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a611d27b0c54f40be9fa0a8ac450719" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 35911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5a41d7f2624bd991f00537c8543110" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 35161 } health_report { overall_health: HEALTHY } } }
W20260811 19:11:41.081825 12003 tablet.cc:2369] T 3b155fd608be48ceab75173c660a66c2 P 6a611d27b0c54f40be9fa0a8ac450719: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:11:41.091074 12034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:41.085673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41880 (local address 127.3.61.130:35161)
0811 19:11:41.085885 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:41.085891 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:41.085913 (+ 22us) server_negotiation.cc:407] Connection header received
0811 19:11:41.086165 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:41.086169 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:41.086259 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:41.086409 (+ 150us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:11:41.086793 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:41.087513 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:41.088863 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:41.089284 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:41.089405 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:41.089914 (+ 509us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:11:41.090458 (+ 544us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:11:41.090617 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:41.090697 (+ 80us) server_negotiation.cc:299] Negotiation successful
0811 19:11:41.090785 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 19:11:41.100888 11921 raft_consensus.cc:1261] T 3b155fd608be48ceab75173c660a66c2 P 6a611d27b0c54f40be9fa0a8ac450719 [term 1 FOLLOWER]: Refusing update from remote peer ee5a41d7f2624bd991f00537c8543110: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:11:41.101016 11435 raft_consensus.cc:1261] T 3b155fd608be48ceab75173c660a66c2 P 54f6dd22a77b4c4e8ab990b649081ccc [term 1 FOLLOWER]: Refusing update from remote peer ee5a41d7f2624bd991f00537c8543110: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:11:41.102378 12059 consensus_queue.cc:1035] T 3b155fd608be48ceab75173c660a66c2 P ee5a41d7f2624bd991f00537c8543110 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a611d27b0c54f40be9fa0a8ac450719" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 35911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:11:41.102818 12054 consensus_queue.cc:1035] T 3b155fd608be48ceab75173c660a66c2 P ee5a41d7f2624bd991f00537c8543110 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54f6dd22a77b4c4e8ab990b649081ccc" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 35399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:11:41.135607 11206 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53136:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:11:41.136044 11206 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53136:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:11:41.140241 11206 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a171d8157c744665970dcf334f8c8494: Sending DeleteTablet for 3 replicas of tablet 3b155fd608be48ceab75173c660a66c2
I20260811 19:11:41.141546 11639 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b155fd608be48ceab75173c660a66c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:11:41 UTC) from {username='kudu', principal='kudu/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:41866
I20260811 19:11:41.141848 11407 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b155fd608be48ceab75173c660a66c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:11:41 UTC) from {username='kudu', principal='kudu/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:34214
I20260811 19:11:41.141950 11880 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b155fd608be48ceab75173c660a66c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:11:41 UTC) from {username='kudu', principal='kudu/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:49000
I20260811 19:11:41.144253 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 11306
I20260811 19:11:41.154260 12083 tablet_replica.cc:330] T 3b155fd608be48ceab75173c660a66c2 P 6a611d27b0c54f40be9fa0a8ac450719: stopping tablet replica
I20260811 19:11:41.154829 12083 raft_consensus.cc:2229] T 3b155fd608be48ceab75173c660a66c2 P 6a611d27b0c54f40be9fa0a8ac450719 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:11:41.155433 12083 raft_consensus.cc:2258] T 3b155fd608be48ceab75173c660a66c2 P 6a611d27b0c54f40be9fa0a8ac450719 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:11:41.156183 12084 tablet_replica.cc:330] T 3b155fd608be48ceab75173c660a66c2 P 54f6dd22a77b4c4e8ab990b649081ccc: stopping tablet replica
I20260811 19:11:41.156654 12084 raft_consensus.cc:2229] T 3b155fd608be48ceab75173c660a66c2 P 54f6dd22a77b4c4e8ab990b649081ccc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:11:41.157178 12084 raft_consensus.cc:2258] T 3b155fd608be48ceab75173c660a66c2 P 54f6dd22a77b4c4e8ab990b649081ccc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:11:41.160005 12083 ts_tablet_manager.cc:1905] T 3b155fd608be48ceab75173c660a66c2 P 6a611d27b0c54f40be9fa0a8ac450719: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:11:41.168315 12083 ts_tablet_manager.cc:1918] T 3b155fd608be48ceab75173c660a66c2 P 6a611d27b0c54f40be9fa0a8ac450719: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:11:41.168556 12083 log.cc:1196] T 3b155fd608be48ceab75173c660a66c2 P 6a611d27b0c54f40be9fa0a8ac450719: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/ts-2/wal/wals/3b155fd608be48ceab75173c660a66c2
I20260811 19:11:41.169137 12083 ts_tablet_manager.cc:1939] T 3b155fd608be48ceab75173c660a66c2 P 6a611d27b0c54f40be9fa0a8ac450719: Deleting consensus metadata
I20260811 19:11:41.171927 12084 ts_tablet_manager.cc:1905] T 3b155fd608be48ceab75173c660a66c2 P 54f6dd22a77b4c4e8ab990b649081ccc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:11:41.178265 12080 tablet_replica.cc:330] T 3b155fd608be48ceab75173c660a66c2 P ee5a41d7f2624bd991f00537c8543110: stopping tablet replica
I20260811 19:11:41.178963 12080 raft_consensus.cc:2229] T 3b155fd608be48ceab75173c660a66c2 P ee5a41d7f2624bd991f00537c8543110 [term 1 LEADER]: Raft consensus shutting down.
I20260811 19:11:41.179909 12080 raft_consensus.cc:2258] T 3b155fd608be48ceab75173c660a66c2 P ee5a41d7f2624bd991f00537c8543110 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:11:41.182384 11182 catalog_manager.cc:4841] TS 6a611d27b0c54f40be9fa0a8ac450719 (127.3.61.131:35911): tablet 3b155fd608be48ceab75173c660a66c2 (table test-table [id=dcc3fca4fc2a4cb996ecca812440f5bb]) successfully deleted
I20260811 19:11:41.182842 12080 ts_tablet_manager.cc:1905] T 3b155fd608be48ceab75173c660a66c2 P ee5a41d7f2624bd991f00537c8543110: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:11:41.181066 12084 ts_tablet_manager.cc:1918] T 3b155fd608be48ceab75173c660a66c2 P 54f6dd22a77b4c4e8ab990b649081ccc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:11:41.183161 12084 log.cc:1196] T 3b155fd608be48ceab75173c660a66c2 P 54f6dd22a77b4c4e8ab990b649081ccc: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/ts-0/wal/wals/3b155fd608be48ceab75173c660a66c2
I20260811 19:11:41.183749 12084 ts_tablet_manager.cc:1939] T 3b155fd608be48ceab75173c660a66c2 P 54f6dd22a77b4c4e8ab990b649081ccc: Deleting consensus metadata
I20260811 19:11:41.185702 11178 catalog_manager.cc:4841] TS 54f6dd22a77b4c4e8ab990b649081ccc (127.3.61.129:35399): tablet 3b155fd608be48ceab75173c660a66c2 (table test-table [id=dcc3fca4fc2a4cb996ecca812440f5bb]) successfully deleted
Aug 11 19:11:41 dist-test-slave-t7g5 krb5kdc[11128](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475498, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.129@KRBTEST.COM
I20260811 19:11:41.189821 12080 ts_tablet_manager.cc:1918] T 3b155fd608be48ceab75173c660a66c2 P ee5a41d7f2624bd991f00537c8543110: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:11:41.190018 12080 log.cc:1196] T 3b155fd608be48ceab75173c660a66c2 P ee5a41d7f2624bd991f00537c8543110: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786475454804547-3318-0/minicluster-data/ts-1/wal/wals/3b155fd608be48ceab75173c660a66c2
I20260811 19:11:41.190591 12080 ts_tablet_manager.cc:1939] T 3b155fd608be48ceab75173c660a66c2 P ee5a41d7f2624bd991f00537c8543110: Deleting consensus metadata
I20260811 19:11:41.192499 11180 catalog_manager.cc:4841] TS ee5a41d7f2624bd991f00537c8543110 (127.3.61.130:35161): tablet 3b155fd608be48ceab75173c660a66c2 (table test-table [id=dcc3fca4fc2a4cb996ecca812440f5bb]) successfully deleted
I20260811 19:11:41.198000 12030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:41.146412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34220 (local address 127.3.61.129:35399)
0811 19:11:41.146629 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:41.146635 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:41.178112 (+ 31477us) server_negotiation.cc:407] Connection header received
0811 19:11:41.178543 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:41.178550 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:41.178665 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:41.178741 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:41.180833 (+ 2092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:41.182072 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:41.184198 (+ 2126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:41.184861 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:41.184970 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:41.191006 (+ 6036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:41.191027 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:41.191047 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:41.191097 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:41.194831 (+ 3734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:41.195834 (+ 1003us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:41.195842 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:41.195844 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:41.196006 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:41.196456 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:41.196461 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:41.196463 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:41.196889 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:41.197135 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:41.197459 (+ 324us) server_negotiation.cc:299] Negotiation successful
0811 19:11:41.197637 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 19:11:41.198719 11512 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:11:41.311396 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 11306
I20260811 19:11:41.354665 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 11530
Aug 11 19:11:41 dist-test-slave-t7g5 krb5kdc[11128](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475498, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.130@KRBTEST.COM
I20260811 19:11:41.376096 12034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:41.355500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41882 (local address 127.3.61.130:35161)
0811 19:11:41.355688 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:41.355692 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:41.355866 (+ 174us) server_negotiation.cc:407] Connection header received
0811 19:11:41.356015 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:41.356020 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:41.356115 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:41.356229 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:41.357801 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:41.358660 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:41.360522 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:41.361026 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:41.361127 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:41.366323 (+ 5196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:41.366346 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:41.366371 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:41.366428 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:41.371261 (+ 4833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:41.372371 (+ 1110us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:41.372381 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:41.372385 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:41.372585 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:41.373064 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:41.373069 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:41.373072 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:41.373586 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:41.373990 (+ 404us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:41.375209 (+ 1219us) server_negotiation.cc:299] Negotiation successful
0811 19:11:41.375662 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 19:11:41.377403 11749 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:11:41.498737 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 11530
I20260811 19:11:41.532055 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 11768
Aug 11 19:11:41 dist-test-slave-t7g5 krb5kdc[11128](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475498, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.131@KRBTEST.COM
I20260811 19:11:41.552507 12032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:41.533087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49012 (local address 127.3.61.131:35911)
0811 19:11:41.533382 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:41.533391 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:41.533561 (+ 170us) server_negotiation.cc:407] Connection header received
0811 19:11:41.533859 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:41.533866 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:41.533973 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:41.534112 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:41.535548 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:41.536738 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:41.538271 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:41.538935 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:41.539077 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:41.544169 (+ 5092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:41.544194 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:41.544220 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:41.544275 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:41.548810 (+ 4535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:41.549923 (+ 1113us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:41.549934 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:41.549938 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:41.550173 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:41.550667 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:41.550672 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:41.550675 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:41.551197 (+ 522us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:41.551533 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:41.551849 (+ 316us) server_negotiation.cc:299] Negotiation successful
0811 19:11:41.552099 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260811 19:11:41.553310 11994 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:11:41.651048 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 11768
I20260811 19:11:41.673362 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 11152
I20260811 19:11:41.674888 11252 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:11:41.791424 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 11152
2026-08-11T19:11:41Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4940 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-11T19:11:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:11:41Z Disabled control of system clock
I20260811 19:11:41.892383 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:35395
--webserver_interface=127.3.61.190
--webserver_port=0
--builtin_ntp_servers=127.3.61.148:37839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.61.190:35395
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 19:11:42.287642 12097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:11:42.288017 12097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:11:42.288134 12097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:11:42.296787 12097 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:11:42.296989 12097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:11:42.297035 12097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:11:42.297134 12097 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:11:42.297205 12097 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:11:42.308187 12097 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:37839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.61.190:35395
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:35395
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.61.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:11:42.309772 12097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:11:42.312623 12097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:11:42.326097 12108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:42.326066 12111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:42.326138 12113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:11:42.327306 12097 server_base.cc:1034] running on GCE node
I20260811 19:11:42.328249 12097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:11:42.329962 12097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:11:42.331214 12097 hybrid_clock.cc:648] HybridClock initialized: now 1786475502331160 us; error 46 us; skew 500 ppm
I20260811 19:11:42.331610 12097 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 19:11:42.335115 12097 webserver.cc:460] Webserver started at http://127.3.61.190:36041/ using document root <none> and password file <none>
I20260811 19:11:42.336524 12097 fs_manager.cc:359] Metadata directory not provided
I20260811 19:11:42.336732 12097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:11:42.337182 12097 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:11:42.341717 12097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/master-0/data/instance:
uuid: "11a3d172aa65410683842fedaf48daf1"
format_stamp: "Formatted at 2026-08-11 19:11:42 on dist-test-slave-t7g5"
server_key: "b6326bae5cb82954fe6f5d31d84ff35f"
server_key_iv: "bc921956ff41b9e1cb1c5d18edf1d42b"
server_key_version: "encryptionkey@0"
I20260811 19:11:42.342828 12097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/master-0/wal/instance:
uuid: "11a3d172aa65410683842fedaf48daf1"
format_stamp: "Formatted at 2026-08-11 19:11:42 on dist-test-slave-t7g5"
server_key: "b6326bae5cb82954fe6f5d31d84ff35f"
server_key_iv: "bc921956ff41b9e1cb1c5d18edf1d42b"
server_key_version: "encryptionkey@0"
I20260811 19:11:42.351353 12097 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.003s sys 0.005s
I20260811 19:11:42.355967 12129 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:42.358274 12097 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 19:11:42.358579 12097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/master-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/master-0/wal
uuid: "11a3d172aa65410683842fedaf48daf1"
format_stamp: "Formatted at 2026-08-11 19:11:42 on dist-test-slave-t7g5"
server_key: "b6326bae5cb82954fe6f5d31d84ff35f"
server_key_iv: "bc921956ff41b9e1cb1c5d18edf1d42b"
server_key_version: "encryptionkey@0"
I20260811 19:11:42.358906 12097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:11:42.366734 12097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:11:42.367695 12097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:11:42.368032 12097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:11:42.389993 12097 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.190:35395
I20260811 19:11:42.390113 12228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.190:35395 every 8 connection(s)
I20260811 19:11:42.391656 12097 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
I20260811 19:11:42.394049 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 12097
I20260811 19:11:42.394253 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/master-0/wal/instance
I20260811 19:11:42.395052 3318 external_mini_cluster.cc:1428] Setting key 9c1841847692037ed445771bf265d975
I20260811 19:11:42.396585 12229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:11:42.405694 12234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:42.397344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59100 (local address 127.3.61.190:35395)
0811 19:11:42.398535 (+ 1191us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:42.398551 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:42.398605 (+ 54us) server_negotiation.cc:407] Connection header received
0811 19:11:42.400496 (+ 1891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:42.400530 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:42.401274 (+ 744us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:42.401844 (+ 570us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:42.403298 (+ 1454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:42.403308 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:42.403328 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:11:42.403350 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:42.403462 (+ 112us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:11:42.403491 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:42.403767 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:42.403920 (+ 153us) server_negotiation.cc:299] Negotiation successful
0811 19:11:42.404287 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":621,"thread_start_us":383,"threads_started":1}
I20260811 19:11:42.413043 12229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11a3d172aa65410683842fedaf48daf1: Bootstrap starting.
I20260811 19:11:42.417013 12229 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 11a3d172aa65410683842fedaf48daf1: Neither blocks nor log segments found. Creating new log.
I20260811 19:11:42.418277 12229 log.cc:824] T 00000000000000000000000000000000 P 11a3d172aa65410683842fedaf48daf1: Log is configured to *not* fsync() on all Append() calls
I20260811 19:11:42.422312 12229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11a3d172aa65410683842fedaf48daf1: No bootstrap required, opened a new log
I20260811 19:11:42.430083 12229 raft_consensus.cc:348] T 00000000000000000000000000000000 P 11a3d172aa65410683842fedaf48daf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11a3d172aa65410683842fedaf48daf1" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35395 } }
I20260811 19:11:42.430505 12229 raft_consensus.cc:374] T 00000000000000000000000000000000 P 11a3d172aa65410683842fedaf48daf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:11:42.430593 12229 raft_consensus.cc:729] T 00000000000000000000000000000000 P 11a3d172aa65410683842fedaf48daf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11a3d172aa65410683842fedaf48daf1, State: Initialized, Role: FOLLOWER
I20260811 19:11:42.431221 12229 consensus_queue.cc:260] T 00000000000000000000000000000000 P 11a3d172aa65410683842fedaf48daf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11a3d172aa65410683842fedaf48daf1" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35395 } }
I20260811 19:11:42.431644 12229 raft_consensus.cc:388] T 00000000000000000000000000000000 P 11a3d172aa65410683842fedaf48daf1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:11:42.431911 12229 raft_consensus.cc:482] T 00000000000000000000000000000000 P 11a3d172aa65410683842fedaf48daf1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:11:42.432120 12229 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 11a3d172aa65410683842fedaf48daf1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:11:42.434985 12229 raft_consensus.cc:504] T 00000000000000000000000000000000 P 11a3d172aa65410683842fedaf48daf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11a3d172aa65410683842fedaf48daf1" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35395 } }
I20260811 19:11:42.435817 12229 leader_election.cc:302] T 00000000000000000000000000000000 P 11a3d172aa65410683842fedaf48daf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 11a3d172aa65410683842fedaf48daf1; no voters:
I20260811 19:11:42.436573 12229 leader_election.cc:288] T 00000000000000000000000000000000 P 11a3d172aa65410683842fedaf48daf1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:11:42.436771 12236 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 11a3d172aa65410683842fedaf48daf1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:11:42.437669 12236 raft_consensus.cc:686] T 00000000000000000000000000000000 P 11a3d172aa65410683842fedaf48daf1 [term 1 LEADER]: Becoming Leader. State: Replica: 11a3d172aa65410683842fedaf48daf1, State: Running, Role: LEADER
I20260811 19:11:42.438416 12236 consensus_queue.cc:237] T 00000000000000000000000000000000 P 11a3d172aa65410683842fedaf48daf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11a3d172aa65410683842fedaf48daf1" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35395 } }
I20260811 19:11:42.439337 12229 sys_catalog.cc:564] T 00000000000000000000000000000000 P 11a3d172aa65410683842fedaf48daf1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:11:42.441386 12239 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11a3d172aa65410683842fedaf48daf1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11a3d172aa65410683842fedaf48daf1. Latest consensus state: current_term: 1 leader_uuid: "11a3d172aa65410683842fedaf48daf1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11a3d172aa65410683842fedaf48daf1" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35395 } } }
I20260811 19:11:42.441504 12237 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11a3d172aa65410683842fedaf48daf1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "11a3d172aa65410683842fedaf48daf1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11a3d172aa65410683842fedaf48daf1" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35395 } } }
I20260811 19:11:42.441990 12237 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11a3d172aa65410683842fedaf48daf1 [sys.catalog]: This master's current role is: LEADER
I20260811 19:11:42.441992 12239 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11a3d172aa65410683842fedaf48daf1 [sys.catalog]: This master's current role is: LEADER
I20260811 19:11:42.444351 12247 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:11:42.449783 12247 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:11:42.458734 12247 catalog_manager.cc:1347] Generated new cluster ID: e27469e70c5a41cc8d19c88193de7729
I20260811 19:11:42.458882 12247 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:11:42.478075 12247 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:11:42.478644 12247 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:11:42.492082 12247 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 11a3d172aa65410683842fedaf48daf1: Generated new TSK 0
I20260811 19:11:42.493309 12247 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:11:42.500427 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.129:0
--local_ip_for_outbound_sockets=127.3.61.129
--webserver_interface=127.3.61.129
--webserver_port=0
--tserver_master_addrs=127.3.61.190:35395
--builtin_ntp_servers=127.3.61.148:37839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 19:11:42.893365 12265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:11:42.893848 12265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:11:42.893996 12265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:11:42.903860 12265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:11:42.904078 12265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:11:42.904261 12265 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.129
I20260811 19:11:42.915280 12265 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:37839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.61.129
--webserver_port=0
--tserver_master_addrs=127.3.61.190:35395
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.129
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:11:42.916770 12265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:11:42.918735 12265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:11:42.933354 12281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:42.933418 12274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:42.933565 12272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:11:42.934618 12265 server_base.cc:1034] running on GCE node
I20260811 19:11:42.935688 12265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:11:42.937144 12265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:11:42.938548 12265 hybrid_clock.cc:648] HybridClock initialized: now 1786475502938464 us; error 65 us; skew 500 ppm
I20260811 19:11:42.939107 12265 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 19:11:42.941942 12265 webserver.cc:460] Webserver started at http://127.3.61.129:39875/ using document root <none> and password file <none>
I20260811 19:11:42.942749 12265 fs_manager.cc:359] Metadata directory not provided
I20260811 19:11:42.942874 12265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:11:42.943210 12265 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:11:42.946945 12265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/ts-0/data/instance:
uuid: "36d0b91f946e4971b6001d94801ceb65"
format_stamp: "Formatted at 2026-08-11 19:11:42 on dist-test-slave-t7g5"
server_key: "2a39aeb4a132d585787acf3d3719cad5"
server_key_iv: "9bcf32b33c88b5127379d3a92e43509e"
server_key_version: "encryptionkey@0"
I20260811 19:11:42.947942 12265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/ts-0/wal/instance:
uuid: "36d0b91f946e4971b6001d94801ceb65"
format_stamp: "Formatted at 2026-08-11 19:11:42 on dist-test-slave-t7g5"
server_key: "2a39aeb4a132d585787acf3d3719cad5"
server_key_iv: "9bcf32b33c88b5127379d3a92e43509e"
server_key_version: "encryptionkey@0"
I20260811 19:11:42.954418 12265 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260811 19:11:42.960214 12290 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:42.962522 12265 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.007s sys 0.000s
I20260811 19:11:42.962797 12265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/ts-0/wal
uuid: "36d0b91f946e4971b6001d94801ceb65"
format_stamp: "Formatted at 2026-08-11 19:11:42 on dist-test-slave-t7g5"
server_key: "2a39aeb4a132d585787acf3d3719cad5"
server_key_iv: "9bcf32b33c88b5127379d3a92e43509e"
server_key_version: "encryptionkey@0"
I20260811 19:11:42.963125 12265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:11:42.969828 12265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:11:42.970868 12265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:11:42.971167 12265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:11:42.972219 12265 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:11:42.973860 12265 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:11:42.973979 12265 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:42.974095 12265 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:11:42.974176 12265 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:43.011101 12265 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.129:42501
I20260811 19:11:43.011143 12492 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.129:42501 every 8 connection(s)
I20260811 19:11:43.012543 12265 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
I20260811 19:11:43.021756 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 12265
I20260811 19:11:43.022042 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/ts-0/wal/instance
I20260811 19:11:43.022471 12500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:43.015573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:51837 (local address 127.3.61.190:35395)
0811 19:11:43.016096 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:43.016101 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:43.017318 (+ 1217us) server_negotiation.cc:407] Connection header received
0811 19:11:43.019334 (+ 2016us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:43.019340 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:43.019417 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:43.019517 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:43.021368 (+ 1851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:43.021374 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:43.021378 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:11:43.021407 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:43.021471 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:11:43.021486 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:43.021694 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:43.021949 (+ 255us) server_negotiation.cc:299] Negotiation successful
0811 19:11:43.022048 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":258,"threads_started":1}
I20260811 19:11:43.022814 3318 external_mini_cluster.cc:1428] Setting key 0013849e8b18ffaf5250e5171d33e0ff
I20260811 19:11:43.024075 12504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:43.016148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35395 (local address 127.3.61.129:51837)
0811 19:11:43.017129 (+ 981us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:43.017169 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:43.019020 (+ 1851us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:43.019777 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:43.019786 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:43.019889 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:11:43.020779 (+ 890us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 19:11:43.020783 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:11:43.020853 (+ 70us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 19:11:43.021012 (+ 159us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:11:43.021623 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:11:43.021639 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:11:43.021693 (+ 54us) client_negotiation.cc:769] Sending connection context
0811 19:11:43.022068 (+ 375us) client_negotiation.cc:240] Negotiation successful
0811 19:11:43.022524 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":302,"threads_started":1}
I20260811 19:11:43.026247 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.130:0
--local_ip_for_outbound_sockets=127.3.61.130
--webserver_interface=127.3.61.130
--webserver_port=0
--tserver_master_addrs=127.3.61.190:35395
--builtin_ntp_servers=127.3.61.148:37839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 19:11:43.026229 12494 heartbeater.cc:347] Connected to a master server at 127.3.61.190:35395
I20260811 19:11:43.026603 12494 heartbeater.cc:464] Registering TS with master...
I20260811 19:11:43.027264 12494 heartbeater.cc:511] Master 127.3.61.190:35395 requested a full tablet report, sending...
I20260811 19:11:43.030257 12154 ts_manager.cc:194] Registered new tserver with Master: 36d0b91f946e4971b6001d94801ceb65 (127.3.61.129:42501)
W20260811 19:11:43.425414 12505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:11:43.425812 12505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:11:43.425935 12505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:11:43.435987 12505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:11:43.436147 12505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:11:43.436295 12505 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.130
I20260811 19:11:43.448523 12505 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:37839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.61.130
--webserver_port=0
--tserver_master_addrs=127.3.61.190:35395
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.130
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:11:43.451028 12505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:11:43.453715 12505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:11:43.467926 12515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:43.468210 12518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:43.468005 12513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:11:43.469888 12505 server_base.cc:1034] running on GCE node
I20260811 19:11:43.470659 12505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:11:43.471767 12505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:11:43.473109 12505 hybrid_clock.cc:648] HybridClock initialized: now 1786475503473007 us; error 50 us; skew 500 ppm
I20260811 19:11:43.473819 12505 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 19:11:43.476755 12505 webserver.cc:460] Webserver started at http://127.3.61.130:35925/ using document root <none> and password file <none>
I20260811 19:11:43.477557 12505 fs_manager.cc:359] Metadata directory not provided
I20260811 19:11:43.477702 12505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:11:43.478060 12505 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:11:43.481283 12505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/ts-1/data/instance:
uuid: "20f20bedfc744b2db77a2fbb9ef052da"
format_stamp: "Formatted at 2026-08-11 19:11:43 on dist-test-slave-t7g5"
server_key: "ec31441366e117de8e7aa1a56f35e533"
server_key_iv: "b680c969bfe2636d335daceceba08826"
server_key_version: "encryptionkey@0"
I20260811 19:11:43.482288 12505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/ts-1/wal/instance:
uuid: "20f20bedfc744b2db77a2fbb9ef052da"
format_stamp: "Formatted at 2026-08-11 19:11:43 on dist-test-slave-t7g5"
server_key: "ec31441366e117de8e7aa1a56f35e533"
server_key_iv: "b680c969bfe2636d335daceceba08826"
server_key_version: "encryptionkey@0"
I20260811 19:11:43.488090 12505 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20260811 19:11:43.492750 12531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:43.494972 12505 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 19:11:43.495200 12505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/ts-1/wal
uuid: "20f20bedfc744b2db77a2fbb9ef052da"
format_stamp: "Formatted at 2026-08-11 19:11:43 on dist-test-slave-t7g5"
server_key: "ec31441366e117de8e7aa1a56f35e533"
server_key_iv: "b680c969bfe2636d335daceceba08826"
server_key_version: "encryptionkey@0"
I20260811 19:11:43.495496 12505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:11:43.502238 12505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:11:43.503175 12505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:11:43.503456 12505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:11:43.504580 12505 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:11:43.506237 12505 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:11:43.506337 12505 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:43.506456 12505 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:11:43.506536 12505 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:43.542714 12505 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.130:40541
I20260811 19:11:43.542783 12712 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.130:40541 every 8 connection(s)
I20260811 19:11:43.544334 12505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
I20260811 19:11:43.550028 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 12505
I20260811 19:11:43.550209 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/ts-1/wal/instance
I20260811 19:11:43.550661 3318 external_mini_cluster.cc:1428] Setting key c61b6e394ccb3df4a4508b8f451fcf19
I20260811 19:11:43.554250 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.131:0
--local_ip_for_outbound_sockets=127.3.61.131
--webserver_interface=127.3.61.131
--webserver_port=0
--tserver_master_addrs=127.3.61.190:35395
--builtin_ntp_servers=127.3.61.148:37839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 19:11:43.554620 12717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:43.547536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:34351 (local address 127.3.61.190:35395)
0811 19:11:43.548041 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:43.548047 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:43.549579 (+ 1532us) server_negotiation.cc:407] Connection header received
0811 19:11:43.551571 (+ 1992us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:43.551577 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:43.551650 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:43.551764 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:43.553699 (+ 1935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:43.553707 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:43.553713 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:11:43.553738 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:43.553791 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:11:43.553808 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:43.553938 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:43.554221 (+ 283us) server_negotiation.cc:299] Negotiation successful
0811 19:11:43.554284 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":245,"threads_started":1}
I20260811 19:11:43.555950 12718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:43.548320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35395 (local address 127.3.61.130:34351)
0811 19:11:43.549370 (+ 1050us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:43.549417 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:43.551218 (+ 1801us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:43.552045 (+ 827us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:43.552055 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:43.552163 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:11:43.553090 (+ 927us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 19:11:43.553094 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:11:43.553174 (+ 80us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 19:11:43.553315 (+ 141us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:11:43.553928 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:11:43.553939 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:11:43.553948 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 19:11:43.554279 (+ 331us) client_negotiation.cc:240] Negotiation successful
0811 19:11:43.554615 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":325,"threads_started":1}
I20260811 19:11:43.557487 12714 heartbeater.cc:347] Connected to a master server at 127.3.61.190:35395
I20260811 19:11:43.557857 12714 heartbeater.cc:464] Registering TS with master...
I20260811 19:11:43.558699 12714 heartbeater.cc:511] Master 127.3.61.190:35395 requested a full tablet report, sending...
I20260811 19:11:43.560915 12154 ts_manager.cc:194] Registered new tserver with Master: 20f20bedfc744b2db77a2fbb9ef052da (127.3.61.130:40541)
W20260811 19:11:43.941728 12719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:11:43.942093 12719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:11:43.942196 12719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:11:43.950495 12719 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:11:43.950639 12719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:11:43.950798 12719 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.131
I20260811 19:11:43.961035 12719 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:37839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.61.131
--webserver_port=0
--tserver_master_addrs=127.3.61.190:35395
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.131
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:11:43.962579 12719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:11:43.964533 12719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:11:43.979019 12732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:43.979112 12730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:43.979121 12728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:11:43.980089 12719 server_base.cc:1034] running on GCE node
I20260811 19:11:43.980777 12719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:11:43.981956 12719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:11:43.983264 12719 hybrid_clock.cc:648] HybridClock initialized: now 1786475503983151 us; error 79 us; skew 500 ppm
I20260811 19:11:43.983747 12719 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 19:11:43.987077 12719 webserver.cc:460] Webserver started at http://127.3.61.131:32941/ using document root <none> and password file <none>
I20260811 19:11:43.988001 12719 fs_manager.cc:359] Metadata directory not provided
I20260811 19:11:43.988332 12719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:11:43.988814 12719 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:11:43.992266 12719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/ts-2/data/instance:
uuid: "aebfb14f28d74b19bd9d9e8753e58671"
format_stamp: "Formatted at 2026-08-11 19:11:43 on dist-test-slave-t7g5"
server_key: "5977315bb5ea653d2b477a2fcfcb0baf"
server_key_iv: "cdfa836c158baa2471d5ecb344974251"
server_key_version: "encryptionkey@0"
I20260811 19:11:43.993168 12719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/ts-2/wal/instance:
uuid: "aebfb14f28d74b19bd9d9e8753e58671"
format_stamp: "Formatted at 2026-08-11 19:11:43 on dist-test-slave-t7g5"
server_key: "5977315bb5ea653d2b477a2fcfcb0baf"
server_key_iv: "cdfa836c158baa2471d5ecb344974251"
server_key_version: "encryptionkey@0"
I20260811 19:11:43.999864 12719 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260811 19:11:44.005057 12742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:44.007138 12719 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260811 19:11:44.007400 12719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/ts-2/wal
uuid: "aebfb14f28d74b19bd9d9e8753e58671"
format_stamp: "Formatted at 2026-08-11 19:11:43 on dist-test-slave-t7g5"
server_key: "5977315bb5ea653d2b477a2fcfcb0baf"
server_key_iv: "cdfa836c158baa2471d5ecb344974251"
server_key_version: "encryptionkey@0"
I20260811 19:11:44.007694 12719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:11:44.016436 12719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:11:44.017617 12719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:11:44.017987 12719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:11:44.019124 12719 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:11:44.020664 12719 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:11:44.020756 12719 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:44.020850 12719 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:11:44.020905 12719 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:44.033939 12494 heartbeater.cc:503] Master 127.3.61.190:35395 was elected leader, sending a full tablet report...
I20260811 19:11:44.057502 12719 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.131:43977
I20260811 19:11:44.057730 12926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.131:43977 every 8 connection(s)
I20260811 19:11:44.058935 12719 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
I20260811 19:11:44.067164 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 12719
I20260811 19:11:44.067345 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/ts-2/wal/instance
I20260811 19:11:44.067823 3318 external_mini_cluster.cc:1428] Setting key 735d1b719fc04f17016d5005e5e12185
I20260811 19:11:44.068429 12934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:44.061900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:42277 (local address 127.3.61.190:35395)
0811 19:11:44.062423 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:44.062430 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:44.063663 (+ 1233us) server_negotiation.cc:407] Connection header received
0811 19:11:44.065611 (+ 1948us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:44.065618 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:44.065769 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:44.065927 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:44.067531 (+ 1604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:44.067537 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:44.067540 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:11:44.067562 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:44.067614 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:11:44.067628 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:44.067750 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:44.067973 (+ 223us) server_negotiation.cc:299] Negotiation successful
0811 19:11:44.068050 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":257,"threads_started":1}
I20260811 19:11:44.069993 12935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:44.062371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35395 (local address 127.3.61.131:42277)
0811 19:11:44.063469 (+ 1098us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:44.063529 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:44.065225 (+ 1696us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:44.066201 (+ 976us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:44.066210 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:44.066311 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:11:44.067130 (+ 819us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 19:11:44.067133 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:11:44.067178 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 19:11:44.067287 (+ 109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:11:44.067733 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:11:44.067744 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:11:44.067752 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 19:11:44.068052 (+ 300us) client_negotiation.cc:240] Negotiation successful
0811 19:11:44.068280 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":310,"threads_started":1}
I20260811 19:11:44.071415 12931 heartbeater.cc:347] Connected to a master server at 127.3.61.190:35395
I20260811 19:11:44.071798 12931 heartbeater.cc:464] Registering TS with master...
I20260811 19:11:44.072475 12931 heartbeater.cc:511] Master 127.3.61.190:35395 requested a full tablet report, sending...
I20260811 19:11:44.074247 12154 ts_manager.cc:194] Registered new tserver with Master: aebfb14f28d74b19bd9d9e8753e58671 (127.3.61.131:43977)
I20260811 19:11:44.085352 3318 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 19:11:44.093066 12934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:44.090400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33756 (local address 127.3.61.190:35395)
0811 19:11:44.090621 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:44.090626 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:44.091087 (+ 461us) server_negotiation.cc:407] Connection header received
0811 19:11:44.091215 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:44.091219 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:44.091281 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:44.091377 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:44.092406 (+ 1029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:44.092410 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:44.092414 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:11:44.092434 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:44.092469 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:11:44.092484 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:44.092594 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:44.092791 (+ 197us) server_negotiation.cc:299] Negotiation successful
0811 19:11:44.092847 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 19:11:44.098699 12154 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33756:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 19:11:44.107110 12154 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:11:44.152231 12955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:44.122490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59588 (local address 127.3.61.130:40541)
0811 19:11:44.147947 (+ 25457us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:44.147957 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:44.147992 (+ 35us) server_negotiation.cc:407] Connection header received
0811 19:11:44.148072 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:44.148081 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:44.148291 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:44.148453 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:44.151228 (+ 2775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:44.151238 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:44.151255 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:11:44.151277 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:44.151348 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:11:44.151374 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:44.151516 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:44.151806 (+ 290us) server_negotiation.cc:299] Negotiation successful
0811 19:11:44.151937 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25244,"thread_start_us":25098,"threads_started":1}
I20260811 19:11:44.152258 12963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:44.122422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.130:40541 (local address 127.0.0.1:59588)
0811 19:11:44.145396 (+ 22974us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:44.145462 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:44.145716 (+ 254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:44.149405 (+ 3689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:44.149419 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:44.149551 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:11:44.150871 (+ 1320us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 19:11:44.150875 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:11:44.150909 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 19:11:44.151003 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:11:44.151487 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:11:44.151500 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:11:44.151509 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 19:11:44.151708 (+ 199us) client_negotiation.cc:240] Negotiation successful
0811 19:11:44.151806 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22610,"thread_start_us":28310,"threads_started":1}
I20260811 19:11:44.153090 12967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:44.122802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:42501 (local address 127.0.0.1:52500)
0811 19:11:44.146976 (+ 24174us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:44.146998 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:44.147124 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:44.149418 (+ 2294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:44.149424 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:44.149550 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:11:44.150871 (+ 1321us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 19:11:44.150875 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:11:44.150909 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 19:11:44.151003 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:11:44.151630 (+ 627us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:11:44.151635 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:11:44.151640 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 19:11:44.152607 (+ 967us) client_negotiation.cc:240] Negotiation successful
0811 19:11:44.152661 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24001,"thread_start_us":25012,"threads_started":1}
I20260811 19:11:44.153918 12968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:44.123429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52500 (local address 127.3.61.129:42501)
0811 19:11:44.144321 (+ 20892us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:44.144332 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:44.147274 (+ 2942us) server_negotiation.cc:407] Connection header received
0811 19:11:44.147397 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:44.147407 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:44.147682 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:44.147857 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:44.151228 (+ 3371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:44.151238 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:44.151255 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:11:44.151277 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:44.151348 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:11:44.151370 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:44.151548 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:44.153368 (+ 1820us) server_negotiation.cc:299] Negotiation successful
0811 19:11:44.153494 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20601,"thread_start_us":20924,"threads_started":1}
I20260811 19:11:44.156142 12956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:44.122422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.131:43977 (local address 127.0.0.1:34872)
0811 19:11:44.150585 (+ 28163us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:44.150604 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:44.150715 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:44.152630 (+ 1915us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:44.152634 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:44.152659 (+ 25us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:11:44.153824 (+ 1165us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 19:11:44.153828 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:11:44.153845 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 19:11:44.153893 (+ 48us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:11:44.155430 (+ 1537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:11:44.155435 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:11:44.155440 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 19:11:44.155608 (+ 168us) client_negotiation.cc:240] Negotiation successful
0811 19:11:44.155667 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28034,"mutex_wait_us":886,"thread_start_us":23324,"threads_started":1}
I20260811 19:11:44.156828 12964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:44.122981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34872 (local address 127.3.61.131:43977)
0811 19:11:44.146478 (+ 23497us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:44.146489 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:44.151561 (+ 5072us) server_negotiation.cc:407] Connection header received
0811 19:11:44.151664 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:44.151673 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:44.151902 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:44.152059 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:44.155042 (+ 2983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:44.155051 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:44.155066 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:11:44.155086 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:44.155143 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:11:44.155165 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:44.155360 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:44.156356 (+ 996us) server_negotiation.cc:299] Negotiation successful
0811 19:11:44.156468 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23294,"thread_start_us":23091,"threads_started":1}
I20260811 19:11:44.156423 12610 tablet_service.cc:1467] Processing CreateTablet for tablet d30dfc543c48468f9b2c37f86fae4722 (DEFAULT_TABLE table=test-table [id=6185fb228a144156a59ef82d45bd98ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:11:44.157444 12372 tablet_service.cc:1467] Processing CreateTablet for tablet d30dfc543c48468f9b2c37f86fae4722 (DEFAULT_TABLE table=test-table [id=6185fb228a144156a59ef82d45bd98ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:11:44.158598 12610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d30dfc543c48468f9b2c37f86fae4722. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:11:44.158912 12372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d30dfc543c48468f9b2c37f86fae4722. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:11:44.161203 12825 tablet_service.cc:1467] Processing CreateTablet for tablet d30dfc543c48468f9b2c37f86fae4722 (DEFAULT_TABLE table=test-table [id=6185fb228a144156a59ef82d45bd98ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:11:44.162896 12825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d30dfc543c48468f9b2c37f86fae4722. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:11:44.175753 12976 tablet_bootstrap.cc:492] T d30dfc543c48468f9b2c37f86fae4722 P 36d0b91f946e4971b6001d94801ceb65: Bootstrap starting.
I20260811 19:11:44.177875 12973 tablet_bootstrap.cc:492] T d30dfc543c48468f9b2c37f86fae4722 P 20f20bedfc744b2db77a2fbb9ef052da: Bootstrap starting.
I20260811 19:11:44.179697 12976 tablet_bootstrap.cc:654] T d30dfc543c48468f9b2c37f86fae4722 P 36d0b91f946e4971b6001d94801ceb65: Neither blocks nor log segments found. Creating new log.
I20260811 19:11:44.180751 12976 log.cc:824] T d30dfc543c48468f9b2c37f86fae4722 P 36d0b91f946e4971b6001d94801ceb65: Log is configured to *not* fsync() on all Append() calls
I20260811 19:11:44.181923 12973 tablet_bootstrap.cc:654] T d30dfc543c48468f9b2c37f86fae4722 P 20f20bedfc744b2db77a2fbb9ef052da: Neither blocks nor log segments found. Creating new log.
I20260811 19:11:44.183226 12980 tablet_bootstrap.cc:492] T d30dfc543c48468f9b2c37f86fae4722 P aebfb14f28d74b19bd9d9e8753e58671: Bootstrap starting.
I20260811 19:11:44.183372 12973 log.cc:824] T d30dfc543c48468f9b2c37f86fae4722 P 20f20bedfc744b2db77a2fbb9ef052da: Log is configured to *not* fsync() on all Append() calls
I20260811 19:11:44.186751 12976 tablet_bootstrap.cc:492] T d30dfc543c48468f9b2c37f86fae4722 P 36d0b91f946e4971b6001d94801ceb65: No bootstrap required, opened a new log
I20260811 19:11:44.187033 12976 ts_tablet_manager.cc:1397] T d30dfc543c48468f9b2c37f86fae4722 P 36d0b91f946e4971b6001d94801ceb65: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.009s
I20260811 19:11:44.187038 12973 tablet_bootstrap.cc:492] T d30dfc543c48468f9b2c37f86fae4722 P 20f20bedfc744b2db77a2fbb9ef052da: No bootstrap required, opened a new log
I20260811 19:11:44.187237 12980 tablet_bootstrap.cc:654] T d30dfc543c48468f9b2c37f86fae4722 P aebfb14f28d74b19bd9d9e8753e58671: Neither blocks nor log segments found. Creating new log.
I20260811 19:11:44.187393 12973 ts_tablet_manager.cc:1397] T d30dfc543c48468f9b2c37f86fae4722 P 20f20bedfc744b2db77a2fbb9ef052da: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.003s
I20260811 19:11:44.188934 12980 log.cc:824] T d30dfc543c48468f9b2c37f86fae4722 P aebfb14f28d74b19bd9d9e8753e58671: Log is configured to *not* fsync() on all Append() calls
I20260811 19:11:44.193910 12980 tablet_bootstrap.cc:492] T d30dfc543c48468f9b2c37f86fae4722 P aebfb14f28d74b19bd9d9e8753e58671: No bootstrap required, opened a new log
I20260811 19:11:44.194322 12980 ts_tablet_manager.cc:1397] T d30dfc543c48468f9b2c37f86fae4722 P aebfb14f28d74b19bd9d9e8753e58671: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260811 19:11:44.197707 12976 raft_consensus.cc:348] T d30dfc543c48468f9b2c37f86fae4722 P 36d0b91f946e4971b6001d94801ceb65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f20bedfc744b2db77a2fbb9ef052da" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 40541 } } peers { permanent_uuid: "aebfb14f28d74b19bd9d9e8753e58671" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 43977 } } peers { permanent_uuid: "36d0b91f946e4971b6001d94801ceb65" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 42501 } }
I20260811 19:11:44.198402 12976 raft_consensus.cc:374] T d30dfc543c48468f9b2c37f86fae4722 P 36d0b91f946e4971b6001d94801ceb65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:11:44.198568 12976 raft_consensus.cc:729] T d30dfc543c48468f9b2c37f86fae4722 P 36d0b91f946e4971b6001d94801ceb65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d0b91f946e4971b6001d94801ceb65, State: Initialized, Role: FOLLOWER
I20260811 19:11:44.199483 12976 consensus_queue.cc:260] T d30dfc543c48468f9b2c37f86fae4722 P 36d0b91f946e4971b6001d94801ceb65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f20bedfc744b2db77a2fbb9ef052da" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 40541 } } peers { permanent_uuid: "aebfb14f28d74b19bd9d9e8753e58671" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 43977 } } peers { permanent_uuid: "36d0b91f946e4971b6001d94801ceb65" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 42501 } }
I20260811 19:11:44.202052 12973 raft_consensus.cc:348] T d30dfc543c48468f9b2c37f86fae4722 P 20f20bedfc744b2db77a2fbb9ef052da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f20bedfc744b2db77a2fbb9ef052da" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 40541 } } peers { permanent_uuid: "aebfb14f28d74b19bd9d9e8753e58671" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 43977 } } peers { permanent_uuid: "36d0b91f946e4971b6001d94801ceb65" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 42501 } }
I20260811 19:11:44.202925 12973 raft_consensus.cc:374] T d30dfc543c48468f9b2c37f86fae4722 P 20f20bedfc744b2db77a2fbb9ef052da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:11:44.203112 12973 raft_consensus.cc:729] T d30dfc543c48468f9b2c37f86fae4722 P 20f20bedfc744b2db77a2fbb9ef052da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20f20bedfc744b2db77a2fbb9ef052da, State: Initialized, Role: FOLLOWER
I20260811 19:11:44.203999 12973 consensus_queue.cc:260] T d30dfc543c48468f9b2c37f86fae4722 P 20f20bedfc744b2db77a2fbb9ef052da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f20bedfc744b2db77a2fbb9ef052da" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 40541 } } peers { permanent_uuid: "aebfb14f28d74b19bd9d9e8753e58671" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 43977 } } peers { permanent_uuid: "36d0b91f946e4971b6001d94801ceb65" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 42501 } }
I20260811 19:11:44.205962 12980 raft_consensus.cc:348] T d30dfc543c48468f9b2c37f86fae4722 P aebfb14f28d74b19bd9d9e8753e58671 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f20bedfc744b2db77a2fbb9ef052da" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 40541 } } peers { permanent_uuid: "aebfb14f28d74b19bd9d9e8753e58671" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 43977 } } peers { permanent_uuid: "36d0b91f946e4971b6001d94801ceb65" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 42501 } }
I20260811 19:11:44.205627 12714 heartbeater.cc:503] Master 127.3.61.190:35395 was elected leader, sending a full tablet report...
I20260811 19:11:44.206615 12980 raft_consensus.cc:374] T d30dfc543c48468f9b2c37f86fae4722 P aebfb14f28d74b19bd9d9e8753e58671 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:11:44.206780 12980 raft_consensus.cc:729] T d30dfc543c48468f9b2c37f86fae4722 P aebfb14f28d74b19bd9d9e8753e58671 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aebfb14f28d74b19bd9d9e8753e58671, State: Initialized, Role: FOLLOWER
I20260811 19:11:44.206327 12973 ts_tablet_manager.cc:1428] T d30dfc543c48468f9b2c37f86fae4722 P 20f20bedfc744b2db77a2fbb9ef052da: Time spent starting tablet: real 0.019s user 0.011s sys 0.004s
I20260811 19:11:44.208336 12976 ts_tablet_manager.cc:1428] T d30dfc543c48468f9b2c37f86fae4722 P 36d0b91f946e4971b6001d94801ceb65: Time spent starting tablet: real 0.021s user 0.011s sys 0.009s
I20260811 19:11:44.208096 12980 consensus_queue.cc:260] T d30dfc543c48468f9b2c37f86fae4722 P aebfb14f28d74b19bd9d9e8753e58671 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f20bedfc744b2db77a2fbb9ef052da" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 40541 } } peers { permanent_uuid: "aebfb14f28d74b19bd9d9e8753e58671" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 43977 } } peers { permanent_uuid: "36d0b91f946e4971b6001d94801ceb65" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 42501 } }
I20260811 19:11:44.210667 12931 heartbeater.cc:503] Master 127.3.61.190:35395 was elected leader, sending a full tablet report...
I20260811 19:11:44.211548 12980 ts_tablet_manager.cc:1428] T d30dfc543c48468f9b2c37f86fae4722 P aebfb14f28d74b19bd9d9e8753e58671: Time spent starting tablet: real 0.017s user 0.015s sys 0.000s
W20260811 19:11:44.265619 12495 tablet.cc:2369] T d30dfc543c48468f9b2c37f86fae4722 P 36d0b91f946e4971b6001d94801ceb65: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 19:11:44.297024 12715 tablet.cc:2369] T d30dfc543c48468f9b2c37f86fae4722 P 20f20bedfc744b2db77a2fbb9ef052da: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 19:11:44.310684 12933 tablet.cc:2369] T d30dfc543c48468f9b2c37f86fae4722 P aebfb14f28d74b19bd9d9e8753e58671: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:11:44.470147 12988 raft_consensus.cc:482] T d30dfc543c48468f9b2c37f86fae4722 P 20f20bedfc744b2db77a2fbb9ef052da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:11:44.470468 12988 raft_consensus.cc:504] T d30dfc543c48468f9b2c37f86fae4722 P 20f20bedfc744b2db77a2fbb9ef052da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f20bedfc744b2db77a2fbb9ef052da" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 40541 } } peers { permanent_uuid: "aebfb14f28d74b19bd9d9e8753e58671" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 43977 } } peers { permanent_uuid: "36d0b91f946e4971b6001d94801ceb65" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 42501 } }
I20260811 19:11:44.472421 12988 leader_election.cc:288] T d30dfc543c48468f9b2c37f86fae4722 P 20f20bedfc744b2db77a2fbb9ef052da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aebfb14f28d74b19bd9d9e8753e58671 (127.3.61.131:43977), 36d0b91f946e4971b6001d94801ceb65 (127.3.61.129:42501)
I20260811 19:11:44.476101 12964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:44.473023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:47959 (local address 127.3.61.131:43977)
0811 19:11:44.473231 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:44.473237 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:44.473985 (+ 748us) server_negotiation.cc:407] Connection header received
0811 19:11:44.474169 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:44.474173 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:44.474224 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:44.474309 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:44.475497 (+ 1188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:44.475504 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:44.475508 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:11:44.475524 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:44.475556 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:11:44.475569 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:44.475685 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:44.475760 (+ 75us) server_negotiation.cc:299] Negotiation successful
0811 19:11:44.475840 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 19:11:44.476323 12968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:44.473310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:38303 (local address 127.3.61.129:42501)
0811 19:11:44.473623 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:44.473629 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:44.474305 (+ 676us) server_negotiation.cc:407] Connection header received
0811 19:11:44.474459 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:44.474464 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:44.474514 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:44.474598 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:44.475558 (+ 960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:44.475562 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:44.475565 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:11:44.475580 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:44.475609 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:11:44.475620 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:44.475729 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:44.475796 (+ 67us) server_negotiation.cc:299] Negotiation successful
0811 19:11:44.475860 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164}
I20260811 19:11:44.476493 12990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:44.472938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.131:43977 (local address 127.3.61.130:47959)
0811 19:11:44.473866 (+ 928us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:44.473886 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:44.474032 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:44.474363 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:44.474368 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:44.474392 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:11:44.475247 (+ 855us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 19:11:44.475251 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:11:44.475271 (+ 20us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 19:11:44.475327 (+ 56us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:11:44.475658 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:11:44.475663 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:11:44.475668 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 19:11:44.475832 (+ 164us) client_negotiation.cc:240] Negotiation successful
0811 19:11:44.475884 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":807,"thread_start_us":560,"threads_started":1}
I20260811 19:11:44.476832 12991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:44.473203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:42501 (local address 127.3.61.130:38303)
0811 19:11:44.474196 (+ 993us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:44.474209 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:44.474327 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:44.474656 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:44.474661 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:44.474684 (+ 23us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:11:44.475361 (+ 677us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 19:11:44.475363 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:11:44.475376 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 19:11:44.475412 (+ 36us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:11:44.475703 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:11:44.475707 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:11:44.475712 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 19:11:44.475849 (+ 137us) client_negotiation.cc:240] Negotiation successful
0811 19:11:44.475895 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":883,"thread_start_us":637,"threads_started":1}
I20260811 19:11:44.477699 12855 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d30dfc543c48468f9b2c37f86fae4722" candidate_uuid: "20f20bedfc744b2db77a2fbb9ef052da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aebfb14f28d74b19bd9d9e8753e58671" is_pre_election: true
I20260811 19:11:44.478228 12855 raft_consensus.cc:2454] T d30dfc543c48468f9b2c37f86fae4722 P aebfb14f28d74b19bd9d9e8753e58671 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20f20bedfc744b2db77a2fbb9ef052da in term 0.
I20260811 19:11:44.478199 12410 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d30dfc543c48468f9b2c37f86fae4722" candidate_uuid: "20f20bedfc744b2db77a2fbb9ef052da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d0b91f946e4971b6001d94801ceb65" is_pre_election: true
I20260811 19:11:44.478750 12410 raft_consensus.cc:2454] T d30dfc543c48468f9b2c37f86fae4722 P 36d0b91f946e4971b6001d94801ceb65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20f20bedfc744b2db77a2fbb9ef052da in term 0.
I20260811 19:11:44.478884 12536 leader_election.cc:302] T d30dfc543c48468f9b2c37f86fae4722 P 20f20bedfc744b2db77a2fbb9ef052da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20f20bedfc744b2db77a2fbb9ef052da, aebfb14f28d74b19bd9d9e8753e58671; no voters:
I20260811 19:11:44.479389 12988 raft_consensus.cc:2781] T d30dfc543c48468f9b2c37f86fae4722 P 20f20bedfc744b2db77a2fbb9ef052da [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:11:44.479677 12988 raft_consensus.cc:482] T d30dfc543c48468f9b2c37f86fae4722 P 20f20bedfc744b2db77a2fbb9ef052da [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:11:44.479884 12988 raft_consensus.cc:3037] T d30dfc543c48468f9b2c37f86fae4722 P 20f20bedfc744b2db77a2fbb9ef052da [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:11:44.482326 12988 raft_consensus.cc:504] T d30dfc543c48468f9b2c37f86fae4722 P 20f20bedfc744b2db77a2fbb9ef052da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f20bedfc744b2db77a2fbb9ef052da" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 40541 } } peers { permanent_uuid: "aebfb14f28d74b19bd9d9e8753e58671" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 43977 } } peers { permanent_uuid: "36d0b91f946e4971b6001d94801ceb65" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 42501 } }
I20260811 19:11:44.482955 12988 leader_election.cc:288] T d30dfc543c48468f9b2c37f86fae4722 P 20f20bedfc744b2db77a2fbb9ef052da [CANDIDATE]: Term 1 election: Requested vote from peers aebfb14f28d74b19bd9d9e8753e58671 (127.3.61.131:43977), 36d0b91f946e4971b6001d94801ceb65 (127.3.61.129:42501)
I20260811 19:11:44.483490 12855 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d30dfc543c48468f9b2c37f86fae4722" candidate_uuid: "20f20bedfc744b2db77a2fbb9ef052da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aebfb14f28d74b19bd9d9e8753e58671"
I20260811 19:11:44.483502 12410 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d30dfc543c48468f9b2c37f86fae4722" candidate_uuid: "20f20bedfc744b2db77a2fbb9ef052da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d0b91f946e4971b6001d94801ceb65"
I20260811 19:11:44.483721 12855 raft_consensus.cc:3037] T d30dfc543c48468f9b2c37f86fae4722 P aebfb14f28d74b19bd9d9e8753e58671 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:11:44.483721 12410 raft_consensus.cc:3037] T d30dfc543c48468f9b2c37f86fae4722 P 36d0b91f946e4971b6001d94801ceb65 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:11:44.487038 12410 raft_consensus.cc:2454] T d30dfc543c48468f9b2c37f86fae4722 P 36d0b91f946e4971b6001d94801ceb65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20f20bedfc744b2db77a2fbb9ef052da in term 1.
I20260811 19:11:44.487038 12855 raft_consensus.cc:2454] T d30dfc543c48468f9b2c37f86fae4722 P aebfb14f28d74b19bd9d9e8753e58671 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20f20bedfc744b2db77a2fbb9ef052da in term 1.
I20260811 19:11:44.487607 12536 leader_election.cc:302] T d30dfc543c48468f9b2c37f86fae4722 P 20f20bedfc744b2db77a2fbb9ef052da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20f20bedfc744b2db77a2fbb9ef052da, aebfb14f28d74b19bd9d9e8753e58671; no voters:
I20260811 19:11:44.487943 12988 raft_consensus.cc:2781] T d30dfc543c48468f9b2c37f86fae4722 P 20f20bedfc744b2db77a2fbb9ef052da [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:11:44.488417 12988 raft_consensus.cc:686] T d30dfc543c48468f9b2c37f86fae4722 P 20f20bedfc744b2db77a2fbb9ef052da [term 1 LEADER]: Becoming Leader. State: Replica: 20f20bedfc744b2db77a2fbb9ef052da, State: Running, Role: LEADER
I20260811 19:11:44.488966 12988 consensus_queue.cc:237] T d30dfc543c48468f9b2c37f86fae4722 P 20f20bedfc744b2db77a2fbb9ef052da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f20bedfc744b2db77a2fbb9ef052da" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 40541 } } peers { permanent_uuid: "aebfb14f28d74b19bd9d9e8753e58671" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 43977 } } peers { permanent_uuid: "36d0b91f946e4971b6001d94801ceb65" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 42501 } }
I20260811 19:11:44.493774 12152 catalog_manager.cc:5495] T d30dfc543c48468f9b2c37f86fae4722 P 20f20bedfc744b2db77a2fbb9ef052da reported cstate change: term changed from 0 to 1, leader changed from <none> to 20f20bedfc744b2db77a2fbb9ef052da (127.3.61.130). New cstate: current_term: 1 leader_uuid: "20f20bedfc744b2db77a2fbb9ef052da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20f20bedfc744b2db77a2fbb9ef052da" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 40541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aebfb14f28d74b19bd9d9e8753e58671" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 43977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36d0b91f946e4971b6001d94801ceb65" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 42501 } health_report { overall_health: UNKNOWN } } }
I20260811 19:11:44.533754 12955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:44.531663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59600 (local address 127.3.61.130:40541)
0811 19:11:44.531841 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:44.531845 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:44.531863 (+ 18us) server_negotiation.cc:407] Connection header received
0811 19:11:44.531974 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:44.531978 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:44.532063 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:44.532146 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:44.533050 (+ 904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:44.533055 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:44.533059 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:11:44.533075 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:44.533116 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:11:44.533135 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:44.533245 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:44.533398 (+ 153us) server_negotiation.cc:299] Negotiation successful
0811 19:11:44.533448 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 19:11:44.546123 12410 raft_consensus.cc:1261] T d30dfc543c48468f9b2c37f86fae4722 P 36d0b91f946e4971b6001d94801ceb65 [term 1 FOLLOWER]: Refusing update from remote peer 20f20bedfc744b2db77a2fbb9ef052da: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:11:44.546099 12855 raft_consensus.cc:1261] T d30dfc543c48468f9b2c37f86fae4722 P aebfb14f28d74b19bd9d9e8753e58671 [term 1 FOLLOWER]: Refusing update from remote peer 20f20bedfc744b2db77a2fbb9ef052da: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:11:44.547680 12988 consensus_queue.cc:1035] T d30dfc543c48468f9b2c37f86fae4722 P 20f20bedfc744b2db77a2fbb9ef052da [LEADER]: Connected to new peer: Peer: permanent_uuid: "aebfb14f28d74b19bd9d9e8753e58671" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 43977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:11:44.548103 12992 consensus_queue.cc:1035] T d30dfc543c48468f9b2c37f86fae4722 P 20f20bedfc744b2db77a2fbb9ef052da [LEADER]: Connected to new peer: Peer: permanent_uuid: "36d0b91f946e4971b6001d94801ceb65" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 42501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:11:44.580351 12154 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33756:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:11:44.580623 12154 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33756:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:11:44.587208 12154 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 11a3d172aa65410683842fedaf48daf1: Sending DeleteTablet for 3 replicas of tablet d30dfc543c48468f9b2c37f86fae4722
I20260811 19:11:44.588671 12825 tablet_service.cc:1514] Processing DeleteTablet for tablet d30dfc543c48468f9b2c37f86fae4722 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:11:44 UTC) from {username='slave'} at 127.0.0.1:34872
I20260811 19:11:44.588717 12610 tablet_service.cc:1514] Processing DeleteTablet for tablet d30dfc543c48468f9b2c37f86fae4722 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:11:44 UTC) from {username='slave'} at 127.0.0.1:59588
I20260811 19:11:44.588848 12372 tablet_service.cc:1514] Processing DeleteTablet for tablet d30dfc543c48468f9b2c37f86fae4722 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:11:44 UTC) from {username='slave'} at 127.0.0.1:52500
I20260811 19:11:44.589742 13017 tablet_replica.cc:330] T d30dfc543c48468f9b2c37f86fae4722 P aebfb14f28d74b19bd9d9e8753e58671: stopping tablet replica
I20260811 19:11:44.590363 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 12265
I20260811 19:11:44.590353 13017 raft_consensus.cc:2229] T d30dfc543c48468f9b2c37f86fae4722 P aebfb14f28d74b19bd9d9e8753e58671 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:11:44.590915 13017 raft_consensus.cc:2258] T d30dfc543c48468f9b2c37f86fae4722 P aebfb14f28d74b19bd9d9e8753e58671 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:11:44.591850 13019 tablet_replica.cc:330] T d30dfc543c48468f9b2c37f86fae4722 P 20f20bedfc744b2db77a2fbb9ef052da: stopping tablet replica
I20260811 19:11:44.592001 13020 tablet_replica.cc:330] T d30dfc543c48468f9b2c37f86fae4722 P 36d0b91f946e4971b6001d94801ceb65: stopping tablet replica
I20260811 19:11:44.592310 13019 raft_consensus.cc:2229] T d30dfc543c48468f9b2c37f86fae4722 P 20f20bedfc744b2db77a2fbb9ef052da [term 1 LEADER]: Raft consensus shutting down.
I20260811 19:11:44.592343 13020 raft_consensus.cc:2229] T d30dfc543c48468f9b2c37f86fae4722 P 36d0b91f946e4971b6001d94801ceb65 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:11:44.592862 13020 raft_consensus.cc:2258] T d30dfc543c48468f9b2c37f86fae4722 P 36d0b91f946e4971b6001d94801ceb65 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:11:44.592984 13019 raft_consensus.cc:2258] T d30dfc543c48468f9b2c37f86fae4722 P 20f20bedfc744b2db77a2fbb9ef052da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:11:44.595731 13020 ts_tablet_manager.cc:1905] T d30dfc543c48468f9b2c37f86fae4722 P 36d0b91f946e4971b6001d94801ceb65: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:11:44.595995 13017 ts_tablet_manager.cc:1905] T d30dfc543c48468f9b2c37f86fae4722 P aebfb14f28d74b19bd9d9e8753e58671: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:11:44.602473 13019 ts_tablet_manager.cc:1905] T d30dfc543c48468f9b2c37f86fae4722 P 20f20bedfc744b2db77a2fbb9ef052da: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:11:44.604969 13017 ts_tablet_manager.cc:1918] T d30dfc543c48468f9b2c37f86fae4722 P aebfb14f28d74b19bd9d9e8753e58671: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:11:44.605350 13017 log.cc:1196] T d30dfc543c48468f9b2c37f86fae4722 P aebfb14f28d74b19bd9d9e8753e58671: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/ts-2/wal/wals/d30dfc543c48468f9b2c37f86fae4722
I20260811 19:11:44.606024 13017 ts_tablet_manager.cc:1939] T d30dfc543c48468f9b2c37f86fae4722 P aebfb14f28d74b19bd9d9e8753e58671: Deleting consensus metadata
I20260811 19:11:44.607508 12968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:44.592793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52514 (local address 127.3.61.129:42501)
0811 19:11:44.592985 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:44.592992 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:44.601009 (+ 8017us) server_negotiation.cc:407] Connection header received
0811 19:11:44.602019 (+ 1010us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:44.602027 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:44.602183 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:44.602275 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:44.606581 (+ 4306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:44.606590 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:44.606595 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:11:44.606628 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:44.606678 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:11:44.606695 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:44.606828 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:44.607104 (+ 276us) server_negotiation.cc:299] Negotiation successful
0811 19:11:44.607181 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 19:11:44.608232 12138 catalog_manager.cc:4841] TS aebfb14f28d74b19bd9d9e8753e58671 (127.3.61.131:43977): tablet d30dfc543c48468f9b2c37f86fae4722 (table test-table [id=6185fb228a144156a59ef82d45bd98ca]) successfully deleted
I20260811 19:11:44.608490 13020 ts_tablet_manager.cc:1918] T d30dfc543c48468f9b2c37f86fae4722 P 36d0b91f946e4971b6001d94801ceb65: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:11:44.608902 12487 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:11:44.609773 13020 log.cc:1196] T d30dfc543c48468f9b2c37f86fae4722 P 36d0b91f946e4971b6001d94801ceb65: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/ts-0/wal/wals/d30dfc543c48468f9b2c37f86fae4722
I20260811 19:11:44.612372 13019 ts_tablet_manager.cc:1918] T d30dfc543c48468f9b2c37f86fae4722 P 20f20bedfc744b2db77a2fbb9ef052da: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:11:44.612622 13019 log.cc:1196] T d30dfc543c48468f9b2c37f86fae4722 P 20f20bedfc744b2db77a2fbb9ef052da: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786475454804547-3318-0/minicluster-data/ts-1/wal/wals/d30dfc543c48468f9b2c37f86fae4722
I20260811 19:11:44.613152 13019 ts_tablet_manager.cc:1939] T d30dfc543c48468f9b2c37f86fae4722 P 20f20bedfc744b2db77a2fbb9ef052da: Deleting consensus metadata
I20260811 19:11:44.614998 12134 catalog_manager.cc:4841] TS 20f20bedfc744b2db77a2fbb9ef052da (127.3.61.130:40541): tablet d30dfc543c48468f9b2c37f86fae4722 (table test-table [id=6185fb228a144156a59ef82d45bd98ca]) successfully deleted
I20260811 19:11:44.756191 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 12265
I20260811 19:11:44.756278 13020 ts_tablet_manager.cc:1939] T d30dfc543c48468f9b2c37f86fae4722 P 36d0b91f946e4971b6001d94801ceb65: Deleting consensus metadata
I20260811 19:11:44.792421 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 12505
W20260811 19:11:44.792375 12137 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.61.129:42501 (error 108)
I20260811 19:11:44.794950 13028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:44.793912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:42501 (local address 127.0.0.1:52516)
0811 19:11:44.794641 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:44.794763 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.129:42501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":612,"thread_start_us":465,"threads_started":1}
W20260811 19:11:44.795241 12137 catalog_manager.cc:4584] TS 36d0b91f946e4971b6001d94801ceb65 (127.3.61.129:42501): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d30dfc543c48468f9b2c37f86fae4722: Network error: Client connection negotiation failed: client connection to 127.3.61.129:42501: connect: Connection refused (error 111)
I20260811 19:11:44.795789 12955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:44.793492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59616 (local address 127.3.61.130:40541)
0811 19:11:44.793688 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:44.793693 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:44.793714 (+ 21us) server_negotiation.cc:407] Connection header received
0811 19:11:44.793799 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:44.793803 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:44.793856 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:44.793964 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:44.794962 (+ 998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:44.794969 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:44.794973 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:11:44.794993 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:44.795035 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:11:44.795055 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:44.795194 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:44.795406 (+ 212us) server_negotiation.cc:299] Negotiation successful
0811 19:11:44.795491 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 19:11:44.796751 12707 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:11:44.849509 13028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:44.848796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:42501 (local address 127.0.0.1:52520)
0811 19:11:44.849151 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:44.849300 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.129:42501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130}
I20260811 19:11:44.850903 13028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:44.850466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:42501 (local address 127.0.0.1:52530)
0811 19:11:44.850622 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:44.850727 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.129:42501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 19:11:44.911669 13028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:44.911165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:42501 (local address 127.0.0.1:52534)
0811 19:11:44.911384 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:44.911495 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.129:42501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260811 19:11:44.912846 13028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:44.912410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:42501 (local address 127.0.0.1:52550)
0811 19:11:44.912580 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:44.912666 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.129:42501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 19:11:44.921195 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 12505
I20260811 19:11:44.946916 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 12719
I20260811 19:11:44.950224 12964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:44.947703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34876 (local address 127.3.61.131:43977)
0811 19:11:44.947876 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:44.947881 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:44.947899 (+ 18us) server_negotiation.cc:407] Connection header received
0811 19:11:44.948040 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:44.948043 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:44.948134 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:44.948217 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:44.949408 (+ 1191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:44.949414 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:44.949417 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:11:44.949434 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:44.949472 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:11:44.949485 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:44.949612 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:44.949808 (+ 196us) server_negotiation.cc:299] Negotiation successful
0811 19:11:44.949864 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 19:11:44.951105 12920 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:11:44.994252 13028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:44.993795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:42501 (local address 127.0.0.1:52560)
0811 19:11:44.994017 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:44.994108 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.129:42501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 19:11:44.995605 13028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:44.995044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:42501 (local address 127.0.0.1:52564)
0811 19:11:44.995258 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:44.995349 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.129:42501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260811 19:11:45.045112 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 12719
I20260811 19:11:45.077646 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 12097
I20260811 19:11:45.079198 12214 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:11:45.196141 13028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:45.195073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:42501 (local address 127.0.0.1:52574)
0811 19:11:45.195625 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:45.195871 (+ 246us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.129:42501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260811 19:11:45.196372 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 12097
2026-08-11T19:11:45Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (3409 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260811 19:11:45.255242 3318 mini_oidc.cc:152] Starting JWKS server
I20260811 19:11:45.258502 3318 webserver.cc:460] Webserver started at https://127.0.0.1:36769/ using document root <none> and password file <none>
I20260811 19:11:45.258864 3318 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 19:11:45.259629 3318 webserver.cc:460] Webserver started at http://127.0.0.1:40129/ using document root <none> and password file <none>
2026-08-11T19:11:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:11:45Z Disabled control of system clock
I20260811 19:11:45.301399 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786475454804547-3318-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786475454804547-3318-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786475454804547-3318-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:43819
--webserver_interface=127.3.61.190
--webserver_port=0
--builtin_ntp_servers=127.3.61.148:37027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.61.190:43819
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36769/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786475454804547-3318-0/testchainca.cert with env {}
W20260811 19:11:45.669878 13044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:11:45.670215 13044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:11:45.670315 13044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:11:45.678930 13044 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:11:45.679065 13044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:11:45.679105 13044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:11:45.679154 13044 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:11:45.679199 13044 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 19:11:45.679268 13044 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 19:11:45.679307 13044 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36769/jwks.json
I20260811 19:11:45.691999 13044 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:37027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786475454804547-3318-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786475454804547-3318-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.61.190:43819
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:43819
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36769/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.61.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786475454804547-3318-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786475454804547-3318-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:11:45.693724 13044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:11:45.695109 13044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:11:45.707101 13051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:45.707080 13053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:45.707134 13055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:11:45.708264 13044 server_base.cc:1034] running on GCE node
I20260811 19:11:45.709712 13044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:11:45.711853 13044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:11:45.713124 13044 hybrid_clock.cc:648] HybridClock initialized: now 1786475505713071 us; error 52 us; skew 500 ppm
I20260811 19:11:45.713639 13044 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 19:11:45.721725 13044 webserver.cc:460] Webserver started at http://127.3.61.190:36539/ using document root <none> and password file <none>
I20260811 19:11:45.722528 13044 fs_manager.cc:359] Metadata directory not provided
I20260811 19:11:45.722652 13044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:11:45.722980 13044 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:11:45.726429 13044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786475454804547-3318-0/minicluster-data/master-0/data/instance:
uuid: "0995682933c748b4b394b3c687b97111"
format_stamp: "Formatted at 2026-08-11 19:11:45 on dist-test-slave-t7g5"
server_key: "b002c818decb23e539e5d3e1b44c23c3"
server_key_iv: "f8ff64874935148969bc4ae4c8b6e4d0"
server_key_version: "encryptionkey@0"
I20260811 19:11:45.727350 13044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786475454804547-3318-0/minicluster-data/master-0/wal/instance:
uuid: "0995682933c748b4b394b3c687b97111"
format_stamp: "Formatted at 2026-08-11 19:11:45 on dist-test-slave-t7g5"
server_key: "b002c818decb23e539e5d3e1b44c23c3"
server_key_iv: "f8ff64874935148969bc4ae4c8b6e4d0"
server_key_version: "encryptionkey@0"
I20260811 19:11:45.733317 13044 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260811 19:11:45.738471 13070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:45.740213 13044 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 19:11:45.740417 13044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786475454804547-3318-0/minicluster-data/master-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786475454804547-3318-0/minicluster-data/master-0/wal
uuid: "0995682933c748b4b394b3c687b97111"
format_stamp: "Formatted at 2026-08-11 19:11:45 on dist-test-slave-t7g5"
server_key: "b002c818decb23e539e5d3e1b44c23c3"
server_key_iv: "f8ff64874935148969bc4ae4c8b6e4d0"
server_key_version: "encryptionkey@0"
I20260811 19:11:45.740661 13044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786475454804547-3318-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786475454804547-3318-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786475454804547-3318-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:11:45.763351 13044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:11:45.764274 13044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:11:45.764602 13044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:11:45.786454 13044 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.190:43819
I20260811 19:11:45.786476 13158 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.190:43819 every 8 connection(s)
I20260811 19:11:45.787865 13044 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
I20260811 19:11:45.791767 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 13044
I20260811 19:11:45.792016 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786475454804547-3318-0/minicluster-data/master-0/wal/instance
I20260811 19:11:45.792157 13159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:11:45.792883 3318 external_mini_cluster.cc:1428] Setting key 9a28e232f4e109cf13cff9cb9e6609e9
I20260811 19:11:45.808456 13159 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0995682933c748b4b394b3c687b97111: Bootstrap starting.
I20260811 19:11:45.811004 13166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:45.795114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59216 (local address 127.3.61.190:43819)
0811 19:11:45.796652 (+ 1538us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:45.796664 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:45.796762 (+ 98us) server_negotiation.cc:407] Connection header received
0811 19:11:45.799697 (+ 2935us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:45.799725 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:45.800234 (+ 509us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:45.800620 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:45.801748 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:45.803036 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:45.804955 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:45.805789 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:45.805962 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:45.807537 (+ 1575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:45.807551 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:45.807576 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:11:45.808369 (+ 793us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:45.808509 (+ 140us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:11:45.808543 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:45.808745 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:45.809033 (+ 288us) server_negotiation.cc:299] Negotiation successful
0811 19:11:45.809559 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":962,"thread_start_us":687,"threads_started":1}
I20260811 19:11:45.812587 13159 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0995682933c748b4b394b3c687b97111: Neither blocks nor log segments found. Creating new log.
I20260811 19:11:45.814150 13159 log.cc:824] T 00000000000000000000000000000000 P 0995682933c748b4b394b3c687b97111: Log is configured to *not* fsync() on all Append() calls
I20260811 19:11:45.817051 3318 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:11:45.818615 13159 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0995682933c748b4b394b3c687b97111: No bootstrap required, opened a new log
I20260811 19:11:45.827105 13159 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0995682933c748b4b394b3c687b97111 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0995682933c748b4b394b3c687b97111" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 43819 } }
I20260811 19:11:45.827518 13159 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0995682933c748b4b394b3c687b97111 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:11:45.827603 13159 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0995682933c748b4b394b3c687b97111 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0995682933c748b4b394b3c687b97111, State: Initialized, Role: FOLLOWER
I20260811 19:11:45.828162 13159 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0995682933c748b4b394b3c687b97111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0995682933c748b4b394b3c687b97111" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 43819 } }
I20260811 19:11:45.828373 13159 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0995682933c748b4b394b3c687b97111 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:11:45.828469 13159 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0995682933c748b4b394b3c687b97111 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:11:45.828606 13159 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0995682933c748b4b394b3c687b97111 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:11:45.831151 13159 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0995682933c748b4b394b3c687b97111 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0995682933c748b4b394b3c687b97111" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 43819 } }
I20260811 19:11:45.831703 13159 leader_election.cc:302] T 00000000000000000000000000000000 P 0995682933c748b4b394b3c687b97111 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0995682933c748b4b394b3c687b97111; no voters:
I20260811 19:11:45.832309 13159 leader_election.cc:288] T 00000000000000000000000000000000 P 0995682933c748b4b394b3c687b97111 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:11:45.832497 13169 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0995682933c748b4b394b3c687b97111 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:11:45.833186 13169 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0995682933c748b4b394b3c687b97111 [term 1 LEADER]: Becoming Leader. State: Replica: 0995682933c748b4b394b3c687b97111, State: Running, Role: LEADER
I20260811 19:11:45.833788 13169 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0995682933c748b4b394b3c687b97111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0995682933c748b4b394b3c687b97111" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 43819 } }
I20260811 19:11:45.834514 13159 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0995682933c748b4b394b3c687b97111 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:11:45.836529 13170 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0995682933c748b4b394b3c687b97111 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0995682933c748b4b394b3c687b97111" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0995682933c748b4b394b3c687b97111" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 43819 } } }
I20260811 19:11:45.837401 13170 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0995682933c748b4b394b3c687b97111 [sys.catalog]: This master's current role is: LEADER
I20260811 19:11:45.836613 13172 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0995682933c748b4b394b3c687b97111 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0995682933c748b4b394b3c687b97111. Latest consensus state: current_term: 1 leader_uuid: "0995682933c748b4b394b3c687b97111" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0995682933c748b4b394b3c687b97111" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 43819 } } }
I20260811 19:11:45.838403 13172 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0995682933c748b4b394b3c687b97111 [sys.catalog]: This master's current role is: LEADER
I20260811 19:11:45.842820 13178 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:11:45.848632 13178 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:11:45.859122 13178 catalog_manager.cc:1347] Generated new cluster ID: 6966fd2e638e4d31b2511c819cb296e0
I20260811 19:11:45.859270 13178 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:11:45.879973 13178 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:11:45.881898 13178 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:11:45.890617 13178 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0995682933c748b4b394b3c687b97111: Generated new TSK 0
I20260811 19:11:45.891541 13178 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 19:11:45.958784 13216 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260811 19:11:45.959048 13216 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 19:11:45.960368 13166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:45.928752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59230 (local address 127.3.61.190:43819)
0811 19:11:45.928977 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:45.928988 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:45.929467 (+ 479us) server_negotiation.cc:407] Connection header received
0811 19:11:45.930155 (+ 688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:45.930160 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:45.930248 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:45.930396 (+ 148us) server_negotiation.cc:226] Negotiated authn=JWT
0811 19:11:45.931107 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:45.931856 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:45.934034 (+ 2178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:45.934595 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:45.934695 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:45.935714 (+ 1019us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 19:11:45.959265 (+ 23551us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36769/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 19:11:45.960053 (+ 788us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59230: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36769/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":94}
W20260811 19:11:45.960541 13166 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59230: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36769/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260811 19:11:45.962620 13214 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:11:45.928515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43819 (local address 127.0.0.1:59230)
0811 19:11:45.929269 (+ 754us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:45.929295 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:45.929768 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:45.930461 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:45.930467 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:45.930481 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0811 19:11:45.930902 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:45.930915 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:45.932009 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:45.932015 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:45.933845 (+ 1830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:45.933861 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:45.934842 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:45.934851 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:45.935095 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:45.935194 (+ 99us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 19:11:45.959862 (+ 24668us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36769/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 19:11:45.960288 (+ 426us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.3.61.190:43819: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36769/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":378,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.3.61.190:43819: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36769/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260811 19:11:45.964836 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 13044
I20260811 19:11:45.966027 13145 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:11:46.071493 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 13044
2026-08-11T19:11:46Z chronyd exiting
I20260811 19:11:46.132882 3318 test_util.cc:175] -----------------------------------------------
I20260811 19:11:46.133062 3318 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786475454804547-3318-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (882 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260811 19:11:46.139616 3318 mini_oidc.cc:152] Starting JWKS server
I20260811 19:11:46.141731 3318 webserver.cc:460] Webserver started at https://127.0.0.1:37927/ using document root <none> and password file <none>
I20260811 19:11:46.141971 3318 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 19:11:46.142732 3318 webserver.cc:460] Webserver started at http://127.0.0.1:36065/ using document root <none> and password file <none>
2026-08-11T19:11:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:11:46Z Disabled control of system clock
I20260811 19:11:46.186831 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786475454804547-3318-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786475454804547-3318-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786475454804547-3318-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:34959
--webserver_interface=127.3.61.190
--webserver_port=0
--builtin_ntp_servers=127.3.61.148:46543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.61.190:34959
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37927/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786475454804547-3318-0/testchainca.cert with env {}
W20260811 19:11:46.591864 13230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:11:46.592345 13230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:11:46.592514 13230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:11:46.601117 13230 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:11:46.601282 13230 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:11:46.601331 13230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:11:46.601395 13230 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:11:46.601476 13230 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 19:11:46.601562 13230 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 19:11:46.601613 13230 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37927/jwks.json
I20260811 19:11:46.613154 13230 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:46543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786475454804547-3318-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786475454804547-3318-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.61.190:34959
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:34959
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37927/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.61.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786475454804547-3318-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786475454804547-3318-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:11:46.614681 13230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:11:46.616086 13230 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:11:46.629252 13236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:46.629323 13240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:46.629297 13237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:11:46.630250 13230 server_base.cc:1034] running on GCE node
I20260811 19:11:46.631125 13230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:11:46.632795 13230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:11:46.634032 13230 hybrid_clock.cc:648] HybridClock initialized: now 1786475506633979 us; error 41 us; skew 500 ppm
I20260811 19:11:46.634438 13230 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 19:11:46.637104 13230 webserver.cc:460] Webserver started at http://127.3.61.190:37717/ using document root <none> and password file <none>
I20260811 19:11:46.638268 13230 fs_manager.cc:359] Metadata directory not provided
I20260811 19:11:46.638516 13230 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:11:46.639097 13230 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:11:46.642316 13230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786475454804547-3318-0/minicluster-data/master-0/data/instance:
uuid: "44b4a8a4dd2140f194658b0367df1c65"
format_stamp: "Formatted at 2026-08-11 19:11:46 on dist-test-slave-t7g5"
server_key: "8ab7f65e89d4e9fec9b8f3ad36877461"
server_key_iv: "449ed52b433dd4b7ce4f1e8104cbfbfa"
server_key_version: "encryptionkey@0"
I20260811 19:11:46.643155 13230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786475454804547-3318-0/minicluster-data/master-0/wal/instance:
uuid: "44b4a8a4dd2140f194658b0367df1c65"
format_stamp: "Formatted at 2026-08-11 19:11:46 on dist-test-slave-t7g5"
server_key: "8ab7f65e89d4e9fec9b8f3ad36877461"
server_key_iv: "449ed52b433dd4b7ce4f1e8104cbfbfa"
server_key_version: "encryptionkey@0"
I20260811 19:11:46.649514 13230 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260811 19:11:46.653467 13249 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:46.655234 13230 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:11:46.655567 13230 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786475454804547-3318-0/minicluster-data/master-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786475454804547-3318-0/minicluster-data/master-0/wal
uuid: "44b4a8a4dd2140f194658b0367df1c65"
format_stamp: "Formatted at 2026-08-11 19:11:46 on dist-test-slave-t7g5"
server_key: "8ab7f65e89d4e9fec9b8f3ad36877461"
server_key_iv: "449ed52b433dd4b7ce4f1e8104cbfbfa"
server_key_version: "encryptionkey@0"
I20260811 19:11:46.655845 13230 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786475454804547-3318-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786475454804547-3318-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786475454804547-3318-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:11:46.695015 13230 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:11:46.696002 13230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:11:46.696336 13230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:11:46.718864 13230 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.190:34959
I20260811 19:11:46.718900 13335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.190:34959 every 8 connection(s)
I20260811 19:11:46.720295 13230 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
I20260811 19:11:46.725807 13337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:11:46.728902 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 13230
I20260811 19:11:46.729135 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786475454804547-3318-0/minicluster-data/master-0/wal/instance
I20260811 19:11:46.729990 3318 external_mini_cluster.cc:1428] Setting key a09ddc74a3fec3d4e392d9871cad5e4b
I20260811 19:11:46.739992 13337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44b4a8a4dd2140f194658b0367df1c65: Bootstrap starting.
I20260811 19:11:46.744014 13337 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44b4a8a4dd2140f194658b0367df1c65: Neither blocks nor log segments found. Creating new log.
I20260811 19:11:46.745448 13337 log.cc:824] T 00000000000000000000000000000000 P 44b4a8a4dd2140f194658b0367df1c65: Log is configured to *not* fsync() on all Append() calls
I20260811 19:11:46.746891 13341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:46.732344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47270 (local address 127.3.61.190:34959)
0811 19:11:46.733592 (+ 1248us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:46.733604 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:46.733694 (+ 90us) server_negotiation.cc:407] Connection header received
0811 19:11:46.735423 (+ 1729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:46.735449 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:46.736003 (+ 554us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:46.736418 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:46.737048 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:46.738475 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:46.739844 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:46.741038 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:46.741352 (+ 314us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:46.743186 (+ 1834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:46.743196 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:46.743220 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:11:46.744107 (+ 887us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:46.744277 (+ 170us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:11:46.744320 (+ 43us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:46.744475 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:46.744768 (+ 293us) server_negotiation.cc:299] Negotiation successful
0811 19:11:46.745245 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":729,"thread_start_us":471,"threads_started":1}
I20260811 19:11:46.749541 13337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44b4a8a4dd2140f194658b0367df1c65: No bootstrap required, opened a new log
I20260811 19:11:46.751807 3318 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:11:46.758811 13337 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44b4a8a4dd2140f194658b0367df1c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44b4a8a4dd2140f194658b0367df1c65" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 34959 } }
I20260811 19:11:46.759255 13337 raft_consensus.cc:374] T 00000000000000000000000000000000 P 44b4a8a4dd2140f194658b0367df1c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:11:46.759369 13337 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44b4a8a4dd2140f194658b0367df1c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44b4a8a4dd2140f194658b0367df1c65, State: Initialized, Role: FOLLOWER
I20260811 19:11:46.760031 13337 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44b4a8a4dd2140f194658b0367df1c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44b4a8a4dd2140f194658b0367df1c65" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 34959 } }
I20260811 19:11:46.760324 13337 raft_consensus.cc:388] T 00000000000000000000000000000000 P 44b4a8a4dd2140f194658b0367df1c65 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:11:46.760480 13337 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44b4a8a4dd2140f194658b0367df1c65 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:11:46.760686 13337 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44b4a8a4dd2140f194658b0367df1c65 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:11:46.763798 13337 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44b4a8a4dd2140f194658b0367df1c65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44b4a8a4dd2140f194658b0367df1c65" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 34959 } }
I20260811 19:11:46.764456 13337 leader_election.cc:302] T 00000000000000000000000000000000 P 44b4a8a4dd2140f194658b0367df1c65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44b4a8a4dd2140f194658b0367df1c65; no voters:
I20260811 19:11:46.765141 13337 leader_election.cc:288] T 00000000000000000000000000000000 P 44b4a8a4dd2140f194658b0367df1c65 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:11:46.765296 13343 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44b4a8a4dd2140f194658b0367df1c65 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:11:46.766772 13343 raft_consensus.cc:686] T 00000000000000000000000000000000 P 44b4a8a4dd2140f194658b0367df1c65 [term 1 LEADER]: Becoming Leader. State: Replica: 44b4a8a4dd2140f194658b0367df1c65, State: Running, Role: LEADER
I20260811 19:11:46.767586 13343 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44b4a8a4dd2140f194658b0367df1c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44b4a8a4dd2140f194658b0367df1c65" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 34959 } }
I20260811 19:11:46.768217 13337 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44b4a8a4dd2140f194658b0367df1c65 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:11:46.770778 13344 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44b4a8a4dd2140f194658b0367df1c65 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "44b4a8a4dd2140f194658b0367df1c65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44b4a8a4dd2140f194658b0367df1c65" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 34959 } } }
I20260811 19:11:46.771291 13344 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44b4a8a4dd2140f194658b0367df1c65 [sys.catalog]: This master's current role is: LEADER
I20260811 19:11:46.771765 13345 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44b4a8a4dd2140f194658b0367df1c65 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44b4a8a4dd2140f194658b0367df1c65. Latest consensus state: current_term: 1 leader_uuid: "44b4a8a4dd2140f194658b0367df1c65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44b4a8a4dd2140f194658b0367df1c65" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 34959 } } }
I20260811 19:11:46.772183 13345 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44b4a8a4dd2140f194658b0367df1c65 [sys.catalog]: This master's current role is: LEADER
I20260811 19:11:46.779069 13358 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:11:46.788877 13358 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:11:46.798631 13358 catalog_manager.cc:1347] Generated new cluster ID: e7b60217c90d4e0ebd636730aa083493
I20260811 19:11:46.798784 13358 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:11:46.831664 13358 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:11:46.833695 13358 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:11:46.850991 13358 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 44b4a8a4dd2140f194658b0367df1c65: Generated new TSK 0
I20260811 19:11:46.851959 13358 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 19:11:46.908206 13395 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260811 19:11:46.908373 13395 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 19:11:46.909337 13341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:46.880896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47274 (local address 127.3.61.190:34959)
0811 19:11:46.881254 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:46.881261 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:46.881530 (+ 269us) server_negotiation.cc:407] Connection header received
0811 19:11:46.881889 (+ 359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:46.881894 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:46.881990 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:46.882152 (+ 162us) server_negotiation.cc:226] Negotiated authn=JWT
0811 19:11:46.882760 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:46.883635 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:46.885763 (+ 2128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:46.886268 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:46.886389 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:46.887046 (+ 657us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 19:11:46.908771 (+ 21725us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37927/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 19:11:46.909111 (+ 340us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47274: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37927/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":78}
W20260811 19:11:46.909485 13341 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47274: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37927/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260811 19:11:46.909817 13394 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:11:46.880396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:34959 (local address 127.0.0.1:47274)
0811 19:11:46.881394 (+ 998us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:46.881415 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:46.881590 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:46.882181 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:46.882187 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:46.882201 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0811 19:11:46.882568 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:46.882582 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:46.883835 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:46.883842 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:46.885557 (+ 1715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:46.885576 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:46.886456 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:46.886463 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:46.886678 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:46.886729 (+ 51us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 19:11:46.909048 (+ 22319us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37927/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 19:11:46.909374 (+ 326us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.3.61.190:34959: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37927/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":831,"thread_start_us":504,"threads_started":1}
I20260811 19:11:46.911056 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 13230
I20260811 19:11:46.913149 13327 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:11:47.014927 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 13230
2026-08-11T19:11:47Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (942 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260811 19:11:47.079496 3318 mini_oidc.cc:152] Starting JWKS server
I20260811 19:11:47.081769 3318 webserver.cc:460] Webserver started at https://127.0.0.1:41299/ using document root <none> and password file <none>
I20260811 19:11:47.082062 3318 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 19:11:47.082854 3318 webserver.cc:460] Webserver started at http://127.0.0.1:44345/ using document root <none> and password file <none>
2026-08-11T19:11:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:11:47Z Disabled control of system clock
I20260811 19:11:47.125979 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786475454804547-3318-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786475454804547-3318-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786475454804547-3318-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:43731
--webserver_interface=127.3.61.190
--webserver_port=0
--builtin_ntp_servers=127.3.61.148:33013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.61.190:43731
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41299/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 19:11:47.508291 13410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:11:47.508985 13410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:11:47.509143 13410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:11:47.518965 13410 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:11:47.519135 13410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:11:47.519176 13410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:11:47.519246 13410 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:11:47.519296 13410 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 19:11:47.519364 13410 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 19:11:47.519405 13410 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41299/jwks.json
I20260811 19:11:47.530387 13410 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:33013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786475454804547-3318-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786475454804547-3318-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.61.190:43731
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:43731
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41299/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.61.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786475454804547-3318-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:11:47.531780 13410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:11:47.533716 13410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:11:47.546818 13419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:47.546885 13422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:47.546856 13431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:11:47.548378 13410 server_base.cc:1034] running on GCE node
I20260811 19:11:47.549773 13410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:11:47.551481 13410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:11:47.552726 13410 hybrid_clock.cc:648] HybridClock initialized: now 1786475507552677 us; error 35 us; skew 500 ppm
I20260811 19:11:47.553097 13410 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 19:11:47.556488 13410 webserver.cc:460] Webserver started at http://127.3.61.190:40423/ using document root <none> and password file <none>
I20260811 19:11:47.557626 13410 fs_manager.cc:359] Metadata directory not provided
I20260811 19:11:47.557806 13410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:11:47.558140 13410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:11:47.561304 13410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786475454804547-3318-0/minicluster-data/master-0/data/instance:
uuid: "fa2e73c259834cc6a3c0ad87411e2911"
format_stamp: "Formatted at 2026-08-11 19:11:47 on dist-test-slave-t7g5"
server_key: "3fab89f79f6b6ef0fa2a6087752c3ec3"
server_key_iv: "9b50799ae97b2e04d36a26ea46754b68"
server_key_version: "encryptionkey@0"
I20260811 19:11:47.562206 13410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786475454804547-3318-0/minicluster-data/master-0/wal/instance:
uuid: "fa2e73c259834cc6a3c0ad87411e2911"
format_stamp: "Formatted at 2026-08-11 19:11:47 on dist-test-slave-t7g5"
server_key: "3fab89f79f6b6ef0fa2a6087752c3ec3"
server_key_iv: "9b50799ae97b2e04d36a26ea46754b68"
server_key_version: "encryptionkey@0"
I20260811 19:11:47.568563 13410 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.005s
I20260811 19:11:47.573244 13441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:47.575450 13410 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260811 19:11:47.575825 13410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786475454804547-3318-0/minicluster-data/master-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786475454804547-3318-0/minicluster-data/master-0/wal
uuid: "fa2e73c259834cc6a3c0ad87411e2911"
format_stamp: "Formatted at 2026-08-11 19:11:47 on dist-test-slave-t7g5"
server_key: "3fab89f79f6b6ef0fa2a6087752c3ec3"
server_key_iv: "9b50799ae97b2e04d36a26ea46754b68"
server_key_version: "encryptionkey@0"
I20260811 19:11:47.576118 13410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786475454804547-3318-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786475454804547-3318-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786475454804547-3318-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:11:47.607688 13410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:11:47.608639 13410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:11:47.608975 13410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:11:47.629711 13410 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.190:43731
I20260811 19:11:47.629796 13524 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.190:43731 every 8 connection(s)
I20260811 19:11:47.631191 13410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
I20260811 19:11:47.635825 13526 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:11:47.639458 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 13410
I20260811 19:11:47.639701 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786475454804547-3318-0/minicluster-data/master-0/wal/instance
I20260811 19:11:47.640522 3318 external_mini_cluster.cc:1428] Setting key 1581a3ddb54144dad0004aad5f0614e9
I20260811 19:11:47.650277 13526 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa2e73c259834cc6a3c0ad87411e2911: Bootstrap starting.
I20260811 19:11:47.655567 13526 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fa2e73c259834cc6a3c0ad87411e2911: Neither blocks nor log segments found. Creating new log.
I20260811 19:11:47.657311 13526 log.cc:824] T 00000000000000000000000000000000 P fa2e73c259834cc6a3c0ad87411e2911: Log is configured to *not* fsync() on all Append() calls
I20260811 19:11:47.660614 13529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:47.643463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59908 (local address 127.3.61.190:43731)
0811 19:11:47.645047 (+ 1584us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:47.645064 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:47.645141 (+ 77us) server_negotiation.cc:407] Connection header received
0811 19:11:47.647560 (+ 2419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:47.647602 (+ 42us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:47.648277 (+ 675us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:47.648701 (+ 424us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:47.649802 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:47.651335 (+ 1533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:47.653811 (+ 2476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:47.654622 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:47.654793 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:47.656611 (+ 1818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:47.656624 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:47.656650 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:11:47.657573 (+ 923us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:47.657774 (+ 201us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:11:47.657809 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:47.657929 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:47.658550 (+ 621us) server_negotiation.cc:299] Negotiation successful
0811 19:11:47.659121 (+ 571us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":811,"thread_start_us":487,"threads_started":1}
I20260811 19:11:47.661808 13526 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa2e73c259834cc6a3c0ad87411e2911: No bootstrap required, opened a new log
I20260811 19:11:47.666438 3318 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:11:47.672451 13526 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa2e73c259834cc6a3c0ad87411e2911 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa2e73c259834cc6a3c0ad87411e2911" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 43731 } }
I20260811 19:11:47.672952 13526 raft_consensus.cc:374] T 00000000000000000000000000000000 P fa2e73c259834cc6a3c0ad87411e2911 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:11:47.673053 13526 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa2e73c259834cc6a3c0ad87411e2911 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa2e73c259834cc6a3c0ad87411e2911, State: Initialized, Role: FOLLOWER
I20260811 19:11:47.673775 13526 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa2e73c259834cc6a3c0ad87411e2911 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa2e73c259834cc6a3c0ad87411e2911" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 43731 } }
I20260811 19:11:47.674053 13526 raft_consensus.cc:388] T 00000000000000000000000000000000 P fa2e73c259834cc6a3c0ad87411e2911 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:11:47.674186 13526 raft_consensus.cc:482] T 00000000000000000000000000000000 P fa2e73c259834cc6a3c0ad87411e2911 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:11:47.674415 13526 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa2e73c259834cc6a3c0ad87411e2911 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:11:47.677430 13526 raft_consensus.cc:504] T 00000000000000000000000000000000 P fa2e73c259834cc6a3c0ad87411e2911 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa2e73c259834cc6a3c0ad87411e2911" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 43731 } }
I20260811 19:11:47.678117 13526 leader_election.cc:302] T 00000000000000000000000000000000 P fa2e73c259834cc6a3c0ad87411e2911 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fa2e73c259834cc6a3c0ad87411e2911; no voters:
I20260811 19:11:47.678915 13526 leader_election.cc:288] T 00000000000000000000000000000000 P fa2e73c259834cc6a3c0ad87411e2911 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:11:47.679021 13532 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fa2e73c259834cc6a3c0ad87411e2911 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:11:47.680277 13532 raft_consensus.cc:686] T 00000000000000000000000000000000 P fa2e73c259834cc6a3c0ad87411e2911 [term 1 LEADER]: Becoming Leader. State: Replica: fa2e73c259834cc6a3c0ad87411e2911, State: Running, Role: LEADER
I20260811 19:11:47.681051 13532 consensus_queue.cc:237] T 00000000000000000000000000000000 P fa2e73c259834cc6a3c0ad87411e2911 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa2e73c259834cc6a3c0ad87411e2911" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 43731 } }
I20260811 19:11:47.681524 13526 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa2e73c259834cc6a3c0ad87411e2911 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:11:47.683815 13535 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa2e73c259834cc6a3c0ad87411e2911 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa2e73c259834cc6a3c0ad87411e2911. Latest consensus state: current_term: 1 leader_uuid: "fa2e73c259834cc6a3c0ad87411e2911" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa2e73c259834cc6a3c0ad87411e2911" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 43731 } } }
I20260811 19:11:47.684238 13535 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa2e73c259834cc6a3c0ad87411e2911 [sys.catalog]: This master's current role is: LEADER
I20260811 19:11:47.684823 13534 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa2e73c259834cc6a3c0ad87411e2911 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fa2e73c259834cc6a3c0ad87411e2911" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa2e73c259834cc6a3c0ad87411e2911" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 43731 } } }
I20260811 19:11:47.685312 13534 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa2e73c259834cc6a3c0ad87411e2911 [sys.catalog]: This master's current role is: LEADER
I20260811 19:11:47.692353 13545 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:11:47.698480 13545 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:11:47.711090 13545 catalog_manager.cc:1347] Generated new cluster ID: 92777e178c164980bd22180d4f056435
I20260811 19:11:47.711263 13545 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:11:47.721264 13545 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:11:47.723110 13545 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:11:47.735735 13545 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fa2e73c259834cc6a3c0ad87411e2911: Generated new TSK 0
I20260811 19:11:47.737521 13545 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:11:47.772028 13529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:47.760512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59910 (local address 127.3.61.190:43731)
0811 19:11:47.760734 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:47.760742 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:47.761180 (+ 438us) server_negotiation.cc:407] Connection header received
0811 19:11:47.761429 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:47.761435 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:47.761544 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:47.761790 (+ 246us) server_negotiation.cc:226] Negotiated authn=JWT
0811 19:11:47.762987 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:47.764136 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:47.766128 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:47.766967 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:47.767203 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:47.768274 (+ 1071us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 19:11:47.771236 (+ 2962us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41299/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 19:11:47.771653 (+ 417us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59910: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41299/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":73}
W20260811 19:11:47.772181 13529 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59910: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41299/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260811 19:11:47.773705 13582 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:11:47.760370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43731 (local address 127.0.0.1:59910)
0811 19:11:47.761066 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:47.761084 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:47.761222 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:47.762313 (+ 1091us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:47.762319 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:47.762338 (+ 19us) client_negotiation.cc:189] Negotiated authn=JWT
0811 19:11:47.762758 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:47.762770 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:47.764385 (+ 1615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:47.764391 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:47.765865 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:47.765887 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:47.767340 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:47.767356 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:47.767696 (+ 340us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:47.767763 (+ 67us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 19:11:47.771559 (+ 3796us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41299/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 19:11:47.773228 (+ 1669us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.3.61.190:43731: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41299/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":552,"mutex_wait_us":1312,"thread_start_us":398,"threads_started":1}
I20260811 19:11:47.775120 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 13410
I20260811 19:11:47.776815 13520 generic_service.cc:196] Checking for leaks (request via RPC)
E20260811 19:11:47.786077 13583 webserver.cc:533] Webserver: sslize failed:
E20260811 19:11:47.786249 13583 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 19:11:47.875669 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 13410
2026-08-11T19:11:47Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (864 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-11T19:11:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:11:47Z Disabled control of system clock
I20260811 19:11:47.986737 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786475454804547-3318-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786475454804547-3318-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786475454804547-3318-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:39265
--webserver_interface=127.3.61.190
--webserver_port=0
--builtin_ntp_servers=127.3.61.148:33117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.61.190:39265
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786475454804547-3318-0/testchainca.cert with env {}
W20260811 19:11:48.358584 13591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:11:48.359184 13591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:11:48.359408 13591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:11:48.368858 13591 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:11:48.369027 13591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:11:48.369067 13591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:11:48.369131 13591 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:11:48.369180 13591 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 19:11:48.369246 13591 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 19:11:48.369287 13591 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260811 19:11:48.381558 13591 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:33117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786475454804547-3318-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786475454804547-3318-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.61.190:39265
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:39265
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.61.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786475454804547-3318-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786475454804547-3318-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:11:48.382948 13591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:11:48.384258 13591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:11:48.397920 13600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:48.398748 13605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:48.398825 13599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:11:48.400180 13591 server_base.cc:1034] running on GCE node
I20260811 19:11:48.401324 13591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:11:48.404968 13591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:11:48.406286 13591 hybrid_clock.cc:648] HybridClock initialized: now 1786475508406227 us; error 49 us; skew 500 ppm
I20260811 19:11:48.406705 13591 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 19:11:48.409379 13591 webserver.cc:460] Webserver started at http://127.3.61.190:43533/ using document root <none> and password file <none>
I20260811 19:11:48.410218 13591 fs_manager.cc:359] Metadata directory not provided
I20260811 19:11:48.410316 13591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:11:48.410665 13591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:11:48.414062 13591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786475454804547-3318-0/minicluster-data/master-0/data/instance:
uuid: "d3b20a8d5c474881bc5b9c5a0704b3eb"
format_stamp: "Formatted at 2026-08-11 19:11:48 on dist-test-slave-t7g5"
server_key: "c874d364d45f2c1d3b13bca8a77cb996"
server_key_iv: "1b3b5150742a9adf4e759a039edb74ca"
server_key_version: "encryptionkey@0"
I20260811 19:11:48.415128 13591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786475454804547-3318-0/minicluster-data/master-0/wal/instance:
uuid: "d3b20a8d5c474881bc5b9c5a0704b3eb"
format_stamp: "Formatted at 2026-08-11 19:11:48 on dist-test-slave-t7g5"
server_key: "c874d364d45f2c1d3b13bca8a77cb996"
server_key_iv: "1b3b5150742a9adf4e759a039edb74ca"
server_key_version: "encryptionkey@0"
I20260811 19:11:48.420732 13591 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260811 19:11:48.424994 13615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:48.426748 13591 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 19:11:48.426964 13591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786475454804547-3318-0/minicluster-data/master-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786475454804547-3318-0/minicluster-data/master-0/wal
uuid: "d3b20a8d5c474881bc5b9c5a0704b3eb"
format_stamp: "Formatted at 2026-08-11 19:11:48 on dist-test-slave-t7g5"
server_key: "c874d364d45f2c1d3b13bca8a77cb996"
server_key_iv: "1b3b5150742a9adf4e759a039edb74ca"
server_key_version: "encryptionkey@0"
I20260811 19:11:48.427243 13591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786475454804547-3318-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786475454804547-3318-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786475454804547-3318-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:11:48.464042 13591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:11:48.464960 13591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:11:48.465246 13591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:11:48.484913 13591 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.190:39265
I20260811 19:11:48.484942 13707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.190:39265 every 8 connection(s)
I20260811 19:11:48.486382 13591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
I20260811 19:11:48.489040 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 13591
I20260811 19:11:48.489288 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786475454804547-3318-0/minicluster-data/master-0/wal/instance
I20260811 19:11:48.490284 3318 external_mini_cluster.cc:1428] Setting key e25ef94efe750637113996828d5693bc
I20260811 19:11:48.490559 13708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:11:48.508844 13708 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3b20a8d5c474881bc5b9c5a0704b3eb: Bootstrap starting.
I20260811 19:11:48.509124 13718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:48.492512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36472 (local address 127.3.61.190:39265)
0811 19:11:48.493686 (+ 1174us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:48.493702 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:48.493772 (+ 70us) server_negotiation.cc:407] Connection header received
0811 19:11:48.495892 (+ 2120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:48.495924 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:48.496915 (+ 991us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:48.497743 (+ 828us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:48.498586 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:48.500895 (+ 2309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:48.502657 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:48.503890 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:48.504137 (+ 247us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:48.505772 (+ 1635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:48.505784 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:48.505809 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:11:48.506614 (+ 805us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:48.506736 (+ 122us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:11:48.506767 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:48.506906 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:48.507200 (+ 294us) server_negotiation.cc:299] Negotiation successful
0811 19:11:48.507601 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":357,"threads_started":1}
I20260811 19:11:48.512447 13708 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d3b20a8d5c474881bc5b9c5a0704b3eb: Neither blocks nor log segments found. Creating new log.
I20260811 19:11:48.513695 13708 log.cc:824] T 00000000000000000000000000000000 P d3b20a8d5c474881bc5b9c5a0704b3eb: Log is configured to *not* fsync() on all Append() calls
I20260811 19:11:48.514930 3318 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:11:48.518113 13708 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3b20a8d5c474881bc5b9c5a0704b3eb: No bootstrap required, opened a new log
I20260811 19:11:48.526779 13708 raft_consensus.cc:348] T 00000000000000000000000000000000 P d3b20a8d5c474881bc5b9c5a0704b3eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3b20a8d5c474881bc5b9c5a0704b3eb" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 39265 } }
I20260811 19:11:48.527261 13708 raft_consensus.cc:374] T 00000000000000000000000000000000 P d3b20a8d5c474881bc5b9c5a0704b3eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:11:48.527380 13708 raft_consensus.cc:729] T 00000000000000000000000000000000 P d3b20a8d5c474881bc5b9c5a0704b3eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3b20a8d5c474881bc5b9c5a0704b3eb, State: Initialized, Role: FOLLOWER
I20260811 19:11:48.528049 13708 consensus_queue.cc:260] T 00000000000000000000000000000000 P d3b20a8d5c474881bc5b9c5a0704b3eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3b20a8d5c474881bc5b9c5a0704b3eb" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 39265 } }
I20260811 19:11:48.528293 13708 raft_consensus.cc:388] T 00000000000000000000000000000000 P d3b20a8d5c474881bc5b9c5a0704b3eb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:11:48.528432 13708 raft_consensus.cc:482] T 00000000000000000000000000000000 P d3b20a8d5c474881bc5b9c5a0704b3eb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:11:48.528605 13708 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d3b20a8d5c474881bc5b9c5a0704b3eb [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:11:48.531251 13708 raft_consensus.cc:504] T 00000000000000000000000000000000 P d3b20a8d5c474881bc5b9c5a0704b3eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3b20a8d5c474881bc5b9c5a0704b3eb" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 39265 } }
I20260811 19:11:48.531864 13708 leader_election.cc:302] T 00000000000000000000000000000000 P d3b20a8d5c474881bc5b9c5a0704b3eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d3b20a8d5c474881bc5b9c5a0704b3eb; no voters:
I20260811 19:11:48.532611 13708 leader_election.cc:288] T 00000000000000000000000000000000 P d3b20a8d5c474881bc5b9c5a0704b3eb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:11:48.532768 13720 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d3b20a8d5c474881bc5b9c5a0704b3eb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:11:48.533538 13720 raft_consensus.cc:686] T 00000000000000000000000000000000 P d3b20a8d5c474881bc5b9c5a0704b3eb [term 1 LEADER]: Becoming Leader. State: Replica: d3b20a8d5c474881bc5b9c5a0704b3eb, State: Running, Role: LEADER
I20260811 19:11:48.534168 13720 consensus_queue.cc:237] T 00000000000000000000000000000000 P d3b20a8d5c474881bc5b9c5a0704b3eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3b20a8d5c474881bc5b9c5a0704b3eb" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 39265 } }
I20260811 19:11:48.534781 13708 sys_catalog.cc:564] T 00000000000000000000000000000000 P d3b20a8d5c474881bc5b9c5a0704b3eb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:11:48.537142 13721 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3b20a8d5c474881bc5b9c5a0704b3eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d3b20a8d5c474881bc5b9c5a0704b3eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3b20a8d5c474881bc5b9c5a0704b3eb" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 39265 } } }
I20260811 19:11:48.537844 13721 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3b20a8d5c474881bc5b9c5a0704b3eb [sys.catalog]: This master's current role is: LEADER
I20260811 19:11:48.538204 13722 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3b20a8d5c474881bc5b9c5a0704b3eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader d3b20a8d5c474881bc5b9c5a0704b3eb. Latest consensus state: current_term: 1 leader_uuid: "d3b20a8d5c474881bc5b9c5a0704b3eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3b20a8d5c474881bc5b9c5a0704b3eb" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 39265 } } }
I20260811 19:11:48.538543 13722 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3b20a8d5c474881bc5b9c5a0704b3eb [sys.catalog]: This master's current role is: LEADER
I20260811 19:11:48.541579 13730 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:11:48.548458 13730 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:11:48.558071 13730 catalog_manager.cc:1347] Generated new cluster ID: 43299c0e29bb4ba58750743cae7ba006
I20260811 19:11:48.558209 13730 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:11:48.596136 13730 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:11:48.598011 13730 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:11:48.606621 13730 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d3b20a8d5c474881bc5b9c5a0704b3eb: Generated new TSK 0
I20260811 19:11:48.607815 13730 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:11:49.087690 13718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:48.624585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36480 (local address 127.3.61.190:39265)
0811 19:11:48.624848 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:48.624855 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:48.625249 (+ 394us) server_negotiation.cc:407] Connection header received
0811 19:11:48.625572 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:48.625578 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:48.625714 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:48.625886 (+ 172us) server_negotiation.cc:226] Negotiated authn=JWT
0811 19:11:48.626566 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:48.627389 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:48.629061 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:48.629540 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:48.629681 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:48.630262 (+ 581us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 19:11:49.086785 (+456523us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 19:11:49.087198 (+ 413us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36480: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":108}
W20260811 19:11:49.087935 13718 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36480: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260811 19:11:49.088116 13758 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:11:48.624256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:39265 (local address 127.0.0.1:36480)
0811 19:11:48.625063 (+ 807us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:48.625087 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:48.625293 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:48.625931 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:48.625937 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:48.625952 (+ 15us) client_negotiation.cc:189] Negotiated authn=JWT
0811 19:11:48.626336 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:48.626350 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:48.627605 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:48.627610 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:48.628880 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:48.628895 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:48.629767 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:48.629774 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:48.629910 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:48.629955 (+ 45us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 19:11:49.087117 (+457162us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 19:11:49.087437 (+ 320us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.3.61.190:39265: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":362,"threads_started":1}
I20260811 19:11:49.089802 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 13591
I20260811 19:11:49.091246 13701 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:11:49.203310 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 13591
2026-08-11T19:11:49Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1313 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (463 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (1571 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:11:51 dist-test-slave-t7g5 krb5kdc[13788](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:11:51 dist-test-slave-t7g5 krb5kdc[13788](info): set up 2 sockets
Aug 11 19:11:51 dist-test-slave-t7g5 krb5kdc[13788](info): commencing operation
krb5kdc: starting...
W20260811 19:11:52.563133 3318 mini_kdc.cc:121] Time spent starting KDC: real 1.270s user 0.005s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Aug 11 19:11:52 dist-test-slave-t7g5 krb5kdc[13788](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475512, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:11:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:11:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.61.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab.
Entry for principal kudu/127.3.61.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab.
I20260811 19:11:52.867511 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:37237
--webserver_interface=127.3.61.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:35165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.61.190:37237
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:11:53.241405 13808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:11:53.242138 13808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:11:53.242343 13808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:11:53.250778 13808 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:11:53.250931 13808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:11:53.250998 13808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:11:53.251096 13808 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:11:53.251149 13808 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:11:53.264535 13808 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:35165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.61.190:37237
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:37237
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.61.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:11:53.266688 13808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:11:53.268186 13808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:11:53.280293 13815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:53.280380 13816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:53.280424 13821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:11:53.281342 13808 server_base.cc:1034] running on GCE node
I20260811 19:11:53.282305 13808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:11:53.283962 13808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:11:53.285341 13808 hybrid_clock.cc:648] HybridClock initialized: now 1786475513285248 us; error 85 us; skew 500 ppm
I20260811 19:11:53.285892 13808 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:11:53 dist-test-slave-t7g5 krb5kdc[13788](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475513, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:11:53.289927 13808 init.cc:377] Logged in from keytab as kudu/127.3.61.190@KRBTEST.COM (short username kudu)
I20260811 19:11:53.291837 13808 webserver.cc:460] Webserver started at http://127.3.61.190:43047/ using document root <none> and password file <none>
I20260811 19:11:53.292654 13808 fs_manager.cc:359] Metadata directory not provided
I20260811 19:11:53.292773 13808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:11:53.293416 13808 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:11:53.296913 13808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/master-0/data/instance:
uuid: "0457ccca72034b2d8973e001920c5ebb"
format_stamp: "Formatted at 2026-08-11 19:11:53 on dist-test-slave-t7g5"
server_key: "40da7693a027d9dee3411e089daa2c13"
server_key_iv: "c7d84957b29f0d71018fba48755fff6a"
server_key_version: "encryptionkey@0"
I20260811 19:11:53.297991 13808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/master-0/wal/instance:
uuid: "0457ccca72034b2d8973e001920c5ebb"
format_stamp: "Formatted at 2026-08-11 19:11:53 on dist-test-slave-t7g5"
server_key: "40da7693a027d9dee3411e089daa2c13"
server_key_iv: "c7d84957b29f0d71018fba48755fff6a"
server_key_version: "encryptionkey@0"
I20260811 19:11:53.305687 13808 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.004s
I20260811 19:11:53.309629 13835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:53.311292 13808 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 19:11:53.311506 13808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/master-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/master-0/wal
uuid: "0457ccca72034b2d8973e001920c5ebb"
format_stamp: "Formatted at 2026-08-11 19:11:53 on dist-test-slave-t7g5"
server_key: "40da7693a027d9dee3411e089daa2c13"
server_key_iv: "c7d84957b29f0d71018fba48755fff6a"
server_key_version: "encryptionkey@0"
I20260811 19:11:53.311724 13808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:11:53.330463 13808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:11:53.335431 13808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:11:53.335965 13808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:11:53.357329 13808 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.190:37237
I20260811 19:11:53.357373 13931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.190:37237 every 8 connection(s)
I20260811 19:11:53.358765 13808 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
I20260811 19:11:53.362620 13934 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:11:53.367617 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 13808
I20260811 19:11:53.367847 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/master-0/wal/instance
I20260811 19:11:53.368669 3318 external_mini_cluster.cc:1428] Setting key 6af05cb98a0df3f4c96b3422b7800639
I20260811 19:11:53.375056 13934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0457ccca72034b2d8973e001920c5ebb: Bootstrap starting.
I20260811 19:11:53.379297 13934 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0457ccca72034b2d8973e001920c5ebb: Neither blocks nor log segments found. Creating new log.
I20260811 19:11:53.380645 13934 log.cc:824] T 00000000000000000000000000000000 P 0457ccca72034b2d8973e001920c5ebb: Log is configured to *not* fsync() on all Append() calls
I20260811 19:11:53.384480 13934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0457ccca72034b2d8973e001920c5ebb: No bootstrap required, opened a new log
Aug 11 19:11:53 dist-test-slave-t7g5 krb5kdc[13788](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475512, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:11:53.392982 13934 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0457ccca72034b2d8973e001920c5ebb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0457ccca72034b2d8973e001920c5ebb" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 37237 } }
I20260811 19:11:53.393392 13934 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0457ccca72034b2d8973e001920c5ebb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:11:53.393494 13934 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0457ccca72034b2d8973e001920c5ebb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0457ccca72034b2d8973e001920c5ebb, State: Initialized, Role: FOLLOWER
I20260811 19:11:53.393959 13940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:53.370870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43346 (local address 127.3.61.190:37237)
0811 19:11:53.371742 (+ 872us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:53.371755 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:53.371801 (+ 46us) server_negotiation.cc:407] Connection header received
0811 19:11:53.373763 (+ 1962us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:53.373791 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:53.374218 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:53.374572 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:53.376603 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:53.378395 (+ 1792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:53.380187 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:53.380990 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:53.381145 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:53.386266 (+ 5121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:53.386279 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:53.386299 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:53.386343 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:53.389623 (+ 3280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:53.390367 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:53.390375 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:53.390382 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:53.390563 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:53.390945 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:53.390949 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:53.390951 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:53.391587 (+ 636us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:53.391890 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:53.392347 (+ 457us) server_negotiation.cc:299] Negotiation successful
0811 19:11:53.392790 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":624,"thread_start_us":377,"threads_started":1}
I20260811 19:11:53.394213 13934 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0457ccca72034b2d8973e001920c5ebb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0457ccca72034b2d8973e001920c5ebb" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 37237 } }
I20260811 19:11:53.394477 13934 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0457ccca72034b2d8973e001920c5ebb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:11:53.394654 13934 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0457ccca72034b2d8973e001920c5ebb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:11:53.394832 13934 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0457ccca72034b2d8973e001920c5ebb [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:11:53.397446 13934 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0457ccca72034b2d8973e001920c5ebb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0457ccca72034b2d8973e001920c5ebb" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 37237 } }
I20260811 19:11:53.398022 13934 leader_election.cc:302] T 00000000000000000000000000000000 P 0457ccca72034b2d8973e001920c5ebb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0457ccca72034b2d8973e001920c5ebb; no voters:
I20260811 19:11:53.398746 13934 leader_election.cc:288] T 00000000000000000000000000000000 P 0457ccca72034b2d8973e001920c5ebb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:11:53.399027 13944 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0457ccca72034b2d8973e001920c5ebb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:11:53.399994 13944 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0457ccca72034b2d8973e001920c5ebb [term 1 LEADER]: Becoming Leader. State: Replica: 0457ccca72034b2d8973e001920c5ebb, State: Running, Role: LEADER
I20260811 19:11:53.400550 13944 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0457ccca72034b2d8973e001920c5ebb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0457ccca72034b2d8973e001920c5ebb" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 37237 } }
I20260811 19:11:53.401180 13934 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0457ccca72034b2d8973e001920c5ebb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:11:53.403546 13945 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0457ccca72034b2d8973e001920c5ebb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0457ccca72034b2d8973e001920c5ebb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0457ccca72034b2d8973e001920c5ebb" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 37237 } } }
I20260811 19:11:53.403520 13946 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0457ccca72034b2d8973e001920c5ebb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0457ccca72034b2d8973e001920c5ebb. Latest consensus state: current_term: 1 leader_uuid: "0457ccca72034b2d8973e001920c5ebb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0457ccca72034b2d8973e001920c5ebb" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 37237 } } }
I20260811 19:11:53.404019 13946 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0457ccca72034b2d8973e001920c5ebb [sys.catalog]: This master's current role is: LEADER
I20260811 19:11:53.404019 13945 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0457ccca72034b2d8973e001920c5ebb [sys.catalog]: This master's current role is: LEADER
I20260811 19:11:53.404997 13956 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:11:53.414403 13956 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:11:53.427372 13956 catalog_manager.cc:1347] Generated new cluster ID: b7cb2b5d10314e42981c096fdf4c867b
I20260811 19:11:53.427525 13956 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:11:53.442559 13956 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:11:53.444311 13956 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:11:53.456880 13956 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0457ccca72034b2d8973e001920c5ebb: Generated new TSK 0
I20260811 19:11:53.458657 13956 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.61.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab.
Entry for principal kudu/127.3.61.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab.
I20260811 19:11:53.545644 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.129:0
--local_ip_for_outbound_sockets=127.3.61.129
--webserver_interface=127.3.61.129
--webserver_port=0
--tserver_master_addrs=127.3.61.190:37237
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab
--principal=kudu/127.3.61.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:35165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:11:53.933202 13983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:11:53.933532 13983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:11:53.933668 13983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:11:53.945335 13983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:11:53.945519 13983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:11:53.945974 13983 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.129
I20260811 19:11:53.958096 13983 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:35165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab
--principal=kudu/127.3.61.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.61.129
--webserver_port=0
--tserver_master_addrs=127.3.61.190:37237
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.129
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:11:53.959542 13983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:11:53.960983 13983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:11:53.976835 13992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:53.976809 13997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:53.976853 13993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:11:53.978077 13983 server_base.cc:1034] running on GCE node
I20260811 19:11:53.978878 13983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:11:53.980166 13983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:11:53.981562 13983 hybrid_clock.cc:648] HybridClock initialized: now 1786475513981483 us; error 63 us; skew 500 ppm
I20260811 19:11:53.982179 13983 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:11:53 dist-test-slave-t7g5 krb5kdc[13788](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475513, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:11:53.985644 13983 init.cc:377] Logged in from keytab as kudu/127.3.61.129@KRBTEST.COM (short username kudu)
I20260811 19:11:53.987522 13983 webserver.cc:460] Webserver started at http://127.3.61.129:42211/ using document root <none> and password file <none>
I20260811 19:11:53.988288 13983 fs_manager.cc:359] Metadata directory not provided
I20260811 19:11:53.988415 13983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:11:53.988763 13983 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:11:53.992018 13983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/ts-0/data/instance:
uuid: "3918c657b1a44dea9bf242cb30a82f64"
format_stamp: "Formatted at 2026-08-11 19:11:53 on dist-test-slave-t7g5"
server_key: "04272a410f150238acfbbd77df2e8cda"
server_key_iv: "a0e8e3d078ed32b316995bd37609cb6c"
server_key_version: "encryptionkey@0"
I20260811 19:11:53.992974 13983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/ts-0/wal/instance:
uuid: "3918c657b1a44dea9bf242cb30a82f64"
format_stamp: "Formatted at 2026-08-11 19:11:53 on dist-test-slave-t7g5"
server_key: "04272a410f150238acfbbd77df2e8cda"
server_key_iv: "a0e8e3d078ed32b316995bd37609cb6c"
server_key_version: "encryptionkey@0"
I20260811 19:11:53.999261 13983 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260811 19:11:54.004612 14007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:54.006795 13983 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260811 19:11:54.007015 13983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/ts-0/wal
uuid: "3918c657b1a44dea9bf242cb30a82f64"
format_stamp: "Formatted at 2026-08-11 19:11:53 on dist-test-slave-t7g5"
server_key: "04272a410f150238acfbbd77df2e8cda"
server_key_iv: "a0e8e3d078ed32b316995bd37609cb6c"
server_key_version: "encryptionkey@0"
I20260811 19:11:54.007258 13983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:11:54.025869 13983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:11:54.031939 13983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:11:54.032222 13983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:11:54.033311 13983 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:11:54.034900 13983 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:11:54.034991 13983 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:54.035090 13983 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:11:54.035135 13983 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:54.074831 13983 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.129:33829
I20260811 19:11:54.074863 14216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.129:33829 every 8 connection(s)
I20260811 19:11:54.076323 13983 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
I20260811 19:11:54.079859 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 13983
I20260811 19:11:54.080137 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/ts-0/wal/instance
I20260811 19:11:54.081202 3318 external_mini_cluster.cc:1428] Setting key 2e0d006b253f281286d1975df504a6f0
Aug 11 19:11:54 dist-test-slave-t7g5 krb5kdc[13788](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475513, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.129@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:11:54.101323 14219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:54.080154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:60395 (local address 127.3.61.190:37237)
0811 19:11:54.080658 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:54.080662 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:54.082469 (+ 1807us) server_negotiation.cc:407] Connection header received
0811 19:11:54.084740 (+ 2271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:54.084745 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:54.084841 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:54.084934 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:54.086662 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:54.087403 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:54.089001 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:54.089430 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:54.089549 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:54.093920 (+ 4371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:54.093927 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:54.093942 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:54.093984 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:54.096863 (+ 2879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:54.098041 (+ 1178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:54.098051 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:54.098055 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:54.098362 (+ 307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:54.098996 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:54.099003 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:54.099005 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:54.099388 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:54.099584 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:54.100686 (+ 1102us) server_negotiation.cc:299] Negotiation successful
0811 19:11:54.100904 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":282,"threads_started":1}
I20260811 19:11:54.102807 14222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:54.081113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:37237 (local address 127.3.61.129:60395)
0811 19:11:54.082281 (+ 1168us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:54.082338 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:54.084456 (+ 2118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:54.085222 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:54.085231 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:54.085855 (+ 624us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:11:54.086485 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:54.086503 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:54.087571 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:54.087575 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:54.088850 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:54.088863 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:54.089823 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:54.089839 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:54.090062 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:54.090843 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:11:54.090870 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:11:54.093587 (+ 2717us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:11:54.097069 (+ 3482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:54.097080 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:54.097104 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:54.097773 (+ 669us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:54.098535 (+ 762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:54.098540 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:54.098545 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:54.098815 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:54.099613 (+ 798us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:11:54.099622 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:11:54.100126 (+ 504us) client_negotiation.cc:769] Sending connection context
0811 19:11:54.100515 (+ 389us) client_negotiation.cc:240] Negotiation successful
0811 19:11:54.101149 (+ 634us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":745,"thread_start_us":470,"threads_started":1}
I20260811 19:11:54.105085 14217 heartbeater.cc:347] Connected to a master server at 127.3.61.190:37237
I20260811 19:11:54.105540 14217 heartbeater.cc:464] Registering TS with master...
I20260811 19:11:54.106668 14217 heartbeater.cc:511] Master 127.3.61.190:37237 requested a full tablet report, sending...
I20260811 19:11:54.109504 13870 ts_manager.cc:194] Registered new tserver with Master: 3918c657b1a44dea9bf242cb30a82f64 (127.3.61.129:33829)
I20260811 19:11:54.111210 13870 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.129@KRBTEST.COM'} at 127.3.61.129:60395
WARNING: no policy specified for kudu/127.3.61.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab.
Entry for principal kudu/127.3.61.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab.
I20260811 19:11:54.153870 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.130:0
--local_ip_for_outbound_sockets=127.3.61.130
--webserver_interface=127.3.61.130
--webserver_port=0
--tserver_master_addrs=127.3.61.190:37237
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab
--principal=kudu/127.3.61.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:35165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:11:54.548758 14226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:11:54.549109 14226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:11:54.549263 14226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:11:54.556946 14226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:11:54.557096 14226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:11:54.557278 14226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.130
I20260811 19:11:54.568967 14226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:35165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab
--principal=kudu/127.3.61.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.61.130
--webserver_port=0
--tserver_master_addrs=127.3.61.190:37237
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.130
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:11:54.570438 14226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:11:54.571792 14226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:11:54.586275 14232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:54.586647 14237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:54.586688 14235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:11:54.588256 14226 server_base.cc:1034] running on GCE node
I20260811 19:11:54.589118 14226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:11:54.590320 14226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:11:54.591609 14226 hybrid_clock.cc:648] HybridClock initialized: now 1786475514591547 us; error 39 us; skew 500 ppm
I20260811 19:11:54.592020 14226 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:11:54 dist-test-slave-t7g5 krb5kdc[13788](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475514, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:11:54.595803 14226 init.cc:377] Logged in from keytab as kudu/127.3.61.130@KRBTEST.COM (short username kudu)
I20260811 19:11:54.597515 14226 webserver.cc:460] Webserver started at http://127.3.61.130:36459/ using document root <none> and password file <none>
I20260811 19:11:54.598286 14226 fs_manager.cc:359] Metadata directory not provided
I20260811 19:11:54.598415 14226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:11:54.598723 14226 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:11:54.602191 14226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/ts-1/data/instance:
uuid: "4971db0ecf794c9288a2d2a1beb6e493"
format_stamp: "Formatted at 2026-08-11 19:11:54 on dist-test-slave-t7g5"
server_key: "dd4e7ed98966d383a6a4f343440d109a"
server_key_iv: "943029ac910c5aac879d06044f7f2cc9"
server_key_version: "encryptionkey@0"
I20260811 19:11:54.603101 14226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/ts-1/wal/instance:
uuid: "4971db0ecf794c9288a2d2a1beb6e493"
format_stamp: "Formatted at 2026-08-11 19:11:54 on dist-test-slave-t7g5"
server_key: "dd4e7ed98966d383a6a4f343440d109a"
server_key_iv: "943029ac910c5aac879d06044f7f2cc9"
server_key_version: "encryptionkey@0"
I20260811 19:11:54.609081 14226 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 19:11:54.613410 14247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:54.614959 14226 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:11:54.615159 14226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/ts-1/wal
uuid: "4971db0ecf794c9288a2d2a1beb6e493"
format_stamp: "Formatted at 2026-08-11 19:11:54 on dist-test-slave-t7g5"
server_key: "dd4e7ed98966d383a6a4f343440d109a"
server_key_iv: "943029ac910c5aac879d06044f7f2cc9"
server_key_version: "encryptionkey@0"
I20260811 19:11:54.615368 14226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:11:54.637553 14226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:11:54.642732 14226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:11:54.643066 14226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:11:54.644244 14226 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:11:54.645836 14226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:11:54.645926 14226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:54.646057 14226 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:11:54.646117 14226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:54.685511 14226 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.130:45073
I20260811 19:11:54.685555 14423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.130:45073 every 8 connection(s)
I20260811 19:11:54.686875 14226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
I20260811 19:11:54.687820 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 14226
I20260811 19:11:54.687999 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/ts-1/wal/instance
I20260811 19:11:54.688596 3318 external_mini_cluster.cc:1428] Setting key f76454f3a34cf9a98c8ed9696e273ab0
Aug 11 19:11:54 dist-test-slave-t7g5 krb5kdc[13788](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475514, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.130@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:11:54.714547 14426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:54.690145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:44335 (local address 127.3.61.190:37237)
0811 19:11:54.690839 (+ 694us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:54.690847 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:54.692194 (+ 1347us) server_negotiation.cc:407] Connection header received
0811 19:11:54.694218 (+ 2024us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:54.694224 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:54.694324 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:54.694424 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:54.696232 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:54.696994 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:54.699268 (+ 2274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:54.699804 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:54.699942 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:54.705482 (+ 5540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:54.705491 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:54.705496 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:54.705553 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:54.709808 (+ 4255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:54.711035 (+ 1227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:54.711041 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:54.711044 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:54.711265 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:54.712112 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:54.712123 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:54.712127 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:54.712535 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:54.712692 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:54.713944 (+ 1252us) server_negotiation.cc:299] Negotiation successful
0811 19:11:54.714197 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":346,"threads_started":1}
I20260811 19:11:54.716157 14427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:54.690804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:37237 (local address 127.3.61.130:44335)
0811 19:11:54.692023 (+ 1219us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:54.692071 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:54.693926 (+ 1855us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:54.694698 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:54.694706 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:54.695266 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:11:54.696022 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:54.696042 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:54.697159 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:54.697163 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:54.698964 (+ 1801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:54.698998 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:54.700119 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:54.700128 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:54.700407 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:54.701330 (+ 923us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:11:54.701368 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:11:54.705089 (+ 3721us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:11:54.710041 (+ 4952us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:54.710051 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:54.710069 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:54.710851 (+ 782us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:54.711420 (+ 569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:54.711426 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:54.711431 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:54.711859 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:54.712741 (+ 882us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:11:54.712750 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:11:54.713367 (+ 617us) client_negotiation.cc:769] Sending connection context
0811 19:11:54.713805 (+ 438us) client_negotiation.cc:240] Negotiation successful
0811 19:11:54.714541 (+ 736us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":730,"thread_start_us":429,"threads_started":1}
I20260811 19:11:54.718075 14424 heartbeater.cc:347] Connected to a master server at 127.3.61.190:37237
I20260811 19:11:54.718468 14424 heartbeater.cc:464] Registering TS with master...
I20260811 19:11:54.719460 14424 heartbeater.cc:511] Master 127.3.61.190:37237 requested a full tablet report, sending...
I20260811 19:11:54.721385 13870 ts_manager.cc:194] Registered new tserver with Master: 4971db0ecf794c9288a2d2a1beb6e493 (127.3.61.130:45073)
I20260811 19:11:54.722955 13870 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.130@KRBTEST.COM'} at 127.3.61.130:44335
WARNING: no policy specified for kudu/127.3.61.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab.
Entry for principal kudu/127.3.61.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab.
I20260811 19:11:54.766363 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.131:0
--local_ip_for_outbound_sockets=127.3.61.131
--webserver_interface=127.3.61.131
--webserver_port=0
--tserver_master_addrs=127.3.61.190:37237
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab
--principal=kudu/127.3.61.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:35165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:11:55.115381 14217 heartbeater.cc:503] Master 127.3.61.190:37237 was elected leader, sending a full tablet report...
W20260811 19:11:55.167423 14431 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:11:55.167775 14431 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:11:55.167912 14431 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:11:55.176651 14431 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:11:55.176807 14431 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:11:55.176980 14431 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.131
I20260811 19:11:55.187929 14431 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:35165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab
--principal=kudu/127.3.61.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.61.131
--webserver_port=0
--tserver_master_addrs=127.3.61.190:37237
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.131
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:11:55.189424 14431 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:11:55.190958 14431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:11:55.206099 14437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:55.206153 14438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:55.206197 14443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:11:55.207050 14431 server_base.cc:1034] running on GCE node
I20260811 19:11:55.207751 14431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:11:55.208827 14431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:11:55.210079 14431 hybrid_clock.cc:648] HybridClock initialized: now 1786475515210032 us; error 36 us; skew 500 ppm
I20260811 19:11:55.210457 14431 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:11:55 dist-test-slave-t7g5 krb5kdc[13788](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475515, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:11:55.214354 14431 init.cc:377] Logged in from keytab as kudu/127.3.61.131@KRBTEST.COM (short username kudu)
I20260811 19:11:55.216202 14431 webserver.cc:460] Webserver started at http://127.3.61.131:35319/ using document root <none> and password file <none>
I20260811 19:11:55.216990 14431 fs_manager.cc:359] Metadata directory not provided
I20260811 19:11:55.217124 14431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:11:55.217437 14431 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:11:55.221104 14431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/ts-2/data/instance:
uuid: "335efdd70be64a889f20420e485e79c5"
format_stamp: "Formatted at 2026-08-11 19:11:55 on dist-test-slave-t7g5"
server_key: "e97b115e3bbd81dff85bbce4b8b25127"
server_key_iv: "20d7a163523a8ba370342cea34a1445d"
server_key_version: "encryptionkey@0"
I20260811 19:11:55.222016 14431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/ts-2/wal/instance:
uuid: "335efdd70be64a889f20420e485e79c5"
format_stamp: "Formatted at 2026-08-11 19:11:55 on dist-test-slave-t7g5"
server_key: "e97b115e3bbd81dff85bbce4b8b25127"
server_key_iv: "20d7a163523a8ba370342cea34a1445d"
server_key_version: "encryptionkey@0"
I20260811 19:11:55.227785 14431 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 19:11:55.232187 14456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:55.234220 14431 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 19:11:55.234493 14431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/ts-2/wal
uuid: "335efdd70be64a889f20420e485e79c5"
format_stamp: "Formatted at 2026-08-11 19:11:55 on dist-test-slave-t7g5"
server_key: "e97b115e3bbd81dff85bbce4b8b25127"
server_key_iv: "20d7a163523a8ba370342cea34a1445d"
server_key_version: "encryptionkey@0"
I20260811 19:11:55.234876 14431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:11:55.260717 14431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:11:55.266346 14431 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:11:55.266664 14431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:11:55.267809 14431 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:11:55.269312 14431 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:11:55.269397 14431 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:55.269521 14431 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:11:55.269575 14431 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:55.305806 14431 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.131:36145
I20260811 19:11:55.305848 14666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.131:36145 every 8 connection(s)
I20260811 19:11:55.307262 14431 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
I20260811 19:11:55.315814 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 14431
I20260811 19:11:55.316012 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/minicluster-data/ts-2/wal/instance
I20260811 19:11:55.316551 3318 external_mini_cluster.cc:1428] Setting key c3513b741197abf5d27196ce92987b0d
Aug 11 19:11:55 dist-test-slave-t7g5 krb5kdc[13788](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475515, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.131@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:11:55.330377 14671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:55.310027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:59643 (local address 127.3.61.190:37237)
0811 19:11:55.310617 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:55.310622 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:55.311708 (+ 1086us) server_negotiation.cc:407] Connection header received
0811 19:11:55.313645 (+ 1937us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:55.313672 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:55.313757 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:55.313853 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:55.315706 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:55.317561 (+ 1855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:55.319590 (+ 2029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:55.320225 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:55.320345 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:55.324217 (+ 3872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:55.324223 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:55.324227 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:55.324265 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:55.326985 (+ 2720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:55.327835 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:55.327840 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:55.327843 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:55.327981 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:55.328435 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:55.328439 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:55.328441 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:55.328733 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:55.328881 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:55.329707 (+ 826us) server_negotiation.cc:299] Negotiation successful
0811 19:11:55.329967 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":273,"threads_started":1}
I20260811 19:11:55.331532 14672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:55.310607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:37237 (local address 127.3.61.131:59643)
0811 19:11:55.311553 (+ 946us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:55.311594 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:55.313364 (+ 1770us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:55.314108 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:55.314117 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:55.314691 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:11:55.315307 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:55.315326 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:55.317773 (+ 2447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:55.317779 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:55.319423 (+ 1644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:55.319441 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:55.320594 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:55.320603 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:55.320855 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:55.321566 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:11:55.321592 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:11:55.323971 (+ 2379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:11:55.327169 (+ 3198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:55.327181 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:55.327200 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:55.327704 (+ 504us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:55.328093 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:55.328098 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:55.328102 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:55.328323 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:55.328874 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:11:55.328882 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:11:55.329297 (+ 415us) client_negotiation.cc:769] Sending connection context
0811 19:11:55.329570 (+ 273us) client_negotiation.cc:240] Negotiation successful
0811 19:11:55.330215 (+ 645us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":650,"thread_start_us":368,"threads_started":1}
I20260811 19:11:55.333138 14667 heartbeater.cc:347] Connected to a master server at 127.3.61.190:37237
I20260811 19:11:55.333467 14667 heartbeater.cc:464] Registering TS with master...
I20260811 19:11:55.334316 14667 heartbeater.cc:511] Master 127.3.61.190:37237 requested a full tablet report, sending...
I20260811 19:11:55.336162 13870 ts_manager.cc:194] Registered new tserver with Master: 335efdd70be64a889f20420e485e79c5 (127.3.61.131:36145)
I20260811 19:11:55.337193 13870 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.131@KRBTEST.COM'} at 127.3.61.131:59643
I20260811 19:11:55.346333 3318 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/krb5kdc/test-admin.keytab.
Aug 11 19:11:55 dist-test-slave-t7g5 krb5kdc[13788](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475515, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:11:55.385687 3318 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260811 19:11:55.391700 14680 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:11:55.389130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36156 (local address 127.3.61.129:33829)
0811 19:11:55.389816 (+ 686us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:55.389824 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:55.389853 (+ 29us) server_negotiation.cc:407] Connection header received
0811 19:11:55.389930 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:55.389939 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:55.390157 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:55.390293 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:55.391276 (+ 983us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36156: BlockingRecv error: recv got EOF from 127.0.0.1:36156 (error 108)
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":341,"threads_started":1}
W20260811 19:11:55.391721 3318 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/krb5kdc/krb5cc' of type FILE
Aug 11 19:11:55 dist-test-slave-t7g5 krb5kdc[13788](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475515, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:11:55.393839 3318 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 19:11:55 dist-test-slave-t7g5 krb5kdc[13788](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475515, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:11:55.395560 3318 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 19:11:55 dist-test-slave-t7g5 krb5kdc[13788](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475515, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.129@KRBTEST.COM
I20260811 19:11:55.415668 14680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:55.397717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36160 (local address 127.3.61.129:33829)
0811 19:11:55.397928 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:55.397936 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:55.397984 (+ 48us) server_negotiation.cc:407] Connection header received
0811 19:11:55.398225 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:55.398231 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:55.398304 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:55.398416 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:55.399667 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:55.401282 (+ 1615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:55.403332 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:55.404078 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:55.404223 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:55.408789 (+ 4566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:55.408809 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:55.408828 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:55.408878 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:55.411985 (+ 3107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:55.412961 (+ 976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:55.412968 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:55.412971 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:55.413126 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:55.413588 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:55.413592 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:55.413594 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:55.414071 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:55.414324 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:55.414726 (+ 402us) server_negotiation.cc:299] Negotiation successful
0811 19:11:55.415022 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260811 19:11:55.423092 14680 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:11:55.420683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36168 (local address 127.3.61.129:33829)
0811 19:11:55.420899 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:55.420905 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:55.421252 (+ 347us) server_negotiation.cc:407] Connection header received
0811 19:11:55.421427 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:55.421431 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:55.421506 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:55.421588 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:55.422878 (+ 1290us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36168: BlockingRecv error: recv got EOF from 127.0.0.1:36168 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260811 19:11:55.423446 3318 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/krb5kdc/krb5cc' of type FILE
Aug 11 19:11:55 dist-test-slave-t7g5 krb5kdc[13788](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475515, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:11:55.425354 3318 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 19:11:55 dist-test-slave-t7g5 krb5kdc[13788](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475515, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:11:55.427299 3318 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 19:11:55 dist-test-slave-t7g5 krb5kdc[13788](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475515, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.129@KRBTEST.COM
I20260811 19:11:55.444796 14680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:55.429153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36170 (local address 127.3.61.129:33829)
0811 19:11:55.429377 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:55.429382 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:55.429588 (+ 206us) server_negotiation.cc:407] Connection header received
0811 19:11:55.429788 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:55.429792 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:55.429867 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:55.429946 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:55.431087 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:55.432113 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:55.433637 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:55.434126 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:55.434239 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:55.438574 (+ 4335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:55.438583 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:55.438588 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:55.438669 (+ 81us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:55.441945 (+ 3276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:55.442715 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:55.442720 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:55.442723 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:55.442901 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:55.443357 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:55.443362 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:55.443364 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:55.443678 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:55.443919 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:55.444271 (+ 352us) server_negotiation.cc:299] Negotiation successful
0811 19:11:55.444488 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
W20260811 19:11:55.450680 14680 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:11:55.448675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36180 (local address 127.3.61.129:33829)
0811 19:11:55.448912 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:55.448918 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:55.449120 (+ 202us) server_negotiation.cc:407] Connection header received
0811 19:11:55.449330 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:55.449334 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:55.449424 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:55.449510 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:55.450457 (+ 947us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36180: BlockingRecv error: recv got EOF from 127.0.0.1:36180 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
W20260811 19:11:55.451292 3318 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786475454804547-3318-0/krb5kdc/krb5cc)
Aug 11 19:11:55 dist-test-slave-t7g5 krb5kdc[13788](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475515, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:11:55.453027 3318 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 19:11:55 dist-test-slave-t7g5 krb5kdc[13788](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475515, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:11:55.454540 3318 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 19:11:55 dist-test-slave-t7g5 krb5kdc[13788](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475515, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.129@KRBTEST.COM
I20260811 19:11:55.472497 14680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:55.456228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36182 (local address 127.3.61.129:33829)
0811 19:11:55.456608 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:55.456617 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:55.456911 (+ 294us) server_negotiation.cc:407] Connection header received
0811 19:11:55.457151 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:55.457156 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:55.457227 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:55.457319 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:55.459020 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:55.459785 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:55.461367 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:55.461897 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:55.462713 (+ 816us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:55.466294 (+ 3581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:55.466300 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:55.466303 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:55.466366 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:55.469584 (+ 3218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:55.470398 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:55.470403 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:55.470405 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:55.470583 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:55.471015 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:55.471019 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:55.471022 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:55.471320 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:55.471561 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:55.471924 (+ 363us) server_negotiation.cc:299] Negotiation successful
0811 19:11:55.472160 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
I20260811 19:11:55.474606 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 13983
I20260811 19:11:55.490588 14680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:55.475482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36198 (local address 127.3.61.129:33829)
0811 19:11:55.476419 (+ 937us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:55.476428 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:55.477177 (+ 749us) server_negotiation.cc:407] Connection header received
0811 19:11:55.477399 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:55.477404 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:55.477501 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:55.477644 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:55.479054 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:55.479800 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:55.481277 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:55.481807 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:55.481915 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:55.484556 (+ 2641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:55.484569 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:55.484574 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:55.484623 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:55.487787 (+ 3164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:55.488555 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:55.488559 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:55.488562 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:55.488702 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:55.489047 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:55.489050 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:55.489052 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:55.489377 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:55.489590 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:55.490023 (+ 433us) server_negotiation.cc:299] Negotiation successful
0811 19:11:55.490254 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":680}
I20260811 19:11:55.491425 14206 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:11:55.578841 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 13983
I20260811 19:11:55.609972 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 14226
Aug 11 19:11:55 dist-test-slave-t7g5 krb5kdc[13788](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475515, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.130@KRBTEST.COM
I20260811 19:11:55.629465 14703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:55.611141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47798 (local address 127.3.61.130:45073)
0811 19:11:55.611739 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:55.611746 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:55.611773 (+ 27us) server_negotiation.cc:407] Connection header received
0811 19:11:55.611837 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:55.611843 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:55.612055 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:55.612191 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:55.613492 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:55.614718 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:55.616363 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:55.617263 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:55.617394 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:55.622155 (+ 4761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:55.622179 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:55.622204 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:55.622282 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:55.626149 (+ 3867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:55.627099 (+ 950us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:55.627108 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:55.627111 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:55.627283 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:55.627700 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:55.627705 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:55.627707 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:55.628125 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:55.628409 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:55.628785 (+ 376us) server_negotiation.cc:299] Negotiation successful
0811 19:11:55.629113 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":294,"threads_started":1}
I20260811 19:11:55.631142 14418 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:11:55.720273 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 14226
I20260811 19:11:55.747546 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 14431
Aug 11 19:11:55 dist-test-slave-t7g5 krb5kdc[13788](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475515, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.131@KRBTEST.COM
I20260811 19:11:55.768801 14706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:55.748886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52586 (local address 127.3.61.131:36145)
0811 19:11:55.749893 (+ 1007us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:55.749903 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:55.749950 (+ 47us) server_negotiation.cc:407] Connection header received
0811 19:11:55.750021 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:55.750028 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:55.750274 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:55.750448 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:55.751642 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:55.752694 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:55.754189 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:55.754801 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:55.754928 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:55.760354 (+ 5426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:55.760378 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:55.760403 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:55.760476 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:55.765382 (+ 4906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:55.766403 (+ 1021us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:55.766412 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:55.766415 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:55.766596 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:55.767011 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:55.767015 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:55.767017 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:55.767436 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:55.767780 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:55.768075 (+ 295us) server_negotiation.cc:299] Negotiation successful
0811 19:11:55.768392 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":395,"threads_started":1}
I20260811 19:11:55.770920 14660 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:11:55.859345 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 14431
I20260811 19:11:55.895134 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 13808
I20260811 19:11:55.896633 13925 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:11:56.012816 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 13808
2026-08-11T19:11:56Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4771 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:11:56 dist-test-slave-t7g5 krb5kdc[14712](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:11:56 dist-test-slave-t7g5 krb5kdc[14712](info): set up 2 sockets
Aug 11 19:11:56 dist-test-slave-t7g5 krb5kdc[14712](info): commencing operation
krb5kdc: starting...
W20260811 19:11:57.348129 3318 mini_kdc.cc:121] Time spent starting KDC: real 1.282s user 0.000s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Aug 11 19:11:57 dist-test-slave-t7g5 krb5kdc[14712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475517, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:11:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:11:57Z Disabled control of system clock
WARNING: no policy specified for oryx/127.3.61.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.61.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.61.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/krb5kdc/oryx_127.3.61.190.keytab.
Entry for principal oryx/127.3.61.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/krb5kdc/oryx_127.3.61.190.keytab.
I20260811 19:11:57.683895 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:33527
--webserver_interface=127.3.61.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/krb5kdc/oryx_127.3.61.190.keytab
--principal=oryx/127.3.61.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:34789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.61.190:33527
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:11:58.087019 14736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:11:58.087466 14736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:11:58.087599 14736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:11:58.096230 14736 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:11:58.096395 14736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:11:58.096460 14736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:11:58.096551 14736 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:11:58.096616 14736 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:11:58.110709 14736 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:34789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.61.190:33527
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/krb5kdc/oryx_127.3.61.190.keytab
--principal=oryx/127.3.61.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:33527
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.61.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:11:58.112282 14736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:11:58.113806 14736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:11:58.128340 14743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:58.128415 14747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:58.128400 14744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:11:58.129387 14736 server_base.cc:1034] running on GCE node
I20260811 19:11:58.130656 14736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:11:58.132822 14736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:11:58.134310 14736 hybrid_clock.cc:648] HybridClock initialized: now 1786475518134224 us; error 61 us; skew 500 ppm
I20260811 19:11:58.134835 14736 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:11:58 dist-test-slave-t7g5 krb5kdc[14712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475518, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.61.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:11:58.138911 14736 init.cc:377] Logged in from keytab as oryx/127.3.61.190@KRBTEST.COM (short username oryx)
I20260811 19:11:58.141239 14736 webserver.cc:460] Webserver started at http://127.3.61.190:39625/ using document root <none> and password file <none>
I20260811 19:11:58.142180 14736 fs_manager.cc:359] Metadata directory not provided
I20260811 19:11:58.142334 14736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:11:58.142874 14736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:11:58.147468 14736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-0/data/instance:
uuid: "d719d69f5e494d2eaecba22f5241c8b4"
format_stamp: "Formatted at 2026-08-11 19:11:58 on dist-test-slave-t7g5"
server_key: "b975f41c9fd42693347a1d06b1b6eaf1"
server_key_iv: "d74d4f4ecbe588c959fd86d261463383"
server_key_version: "encryptionkey@0"
I20260811 19:11:58.148691 14736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-0/wal/instance:
uuid: "d719d69f5e494d2eaecba22f5241c8b4"
format_stamp: "Formatted at 2026-08-11 19:11:58 on dist-test-slave-t7g5"
server_key: "b975f41c9fd42693347a1d06b1b6eaf1"
server_key_iv: "d74d4f4ecbe588c959fd86d261463383"
server_key_version: "encryptionkey@0"
I20260811 19:11:58.156570 14736 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260811 19:11:58.161533 14762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:58.163513 14736 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 19:11:58.163770 14736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-0/wal
uuid: "d719d69f5e494d2eaecba22f5241c8b4"
format_stamp: "Formatted at 2026-08-11 19:11:58 on dist-test-slave-t7g5"
server_key: "b975f41c9fd42693347a1d06b1b6eaf1"
server_key_iv: "d74d4f4ecbe588c959fd86d261463383"
server_key_version: "encryptionkey@0"
I20260811 19:11:58.164019 14736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:11:58.211753 14736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:11:58.217309 14736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:11:58.217832 14736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:11:58.240502 14736 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.190:33527
I20260811 19:11:58.240542 14845 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.190:33527 every 8 connection(s)
I20260811 19:11:58.242228 14736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
I20260811 19:11:58.248422 14846 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:11:58.249090 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 14736
I20260811 19:11:58.249320 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-0/wal/instance
I20260811 19:11:58.250198 3318 external_mini_cluster.cc:1428] Setting key 935fde36b5fe0cb91e50372c9b9cc0db
I20260811 19:11:58.265189 14846 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4: Bootstrap starting.
Aug 11 19:11:58 dist-test-slave-t7g5 krb5kdc[14712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475517, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM
I20260811 19:11:58.269500 14846 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4: Neither blocks nor log segments found. Creating new log.
I20260811 19:11:58.271092 14846 log.cc:824] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4: Log is configured to *not* fsync() on all Append() calls
I20260811 19:11:58.275549 14846 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4: No bootstrap required, opened a new log
I20260811 19:11:58.280180 14850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:58.252722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58344 (local address 127.3.61.190:33527)
0811 19:11:58.253552 (+ 830us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:58.253566 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:58.253617 (+ 51us) server_negotiation.cc:407] Connection header received
0811 19:11:58.255664 (+ 2047us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:58.255694 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:58.256164 (+ 470us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:58.256536 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:58.258729 (+ 2193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:58.260621 (+ 1892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:58.263040 (+ 2419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:58.263881 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:58.264048 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:58.269796 (+ 5748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:58.269812 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:58.269835 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:58.269911 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:58.274219 (+ 4308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:58.275131 (+ 912us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:58.275140 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:58.275150 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:58.275394 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:58.275905 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:58.275910 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:58.275913 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:58.276716 (+ 803us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:58.277076 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:58.277924 (+ 848us) server_negotiation.cc:299] Negotiation successful
0811 19:11:58.278506 (+ 582us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":350,"threads_started":1}
I20260811 19:11:58.286784 14846 raft_consensus.cc:348] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } }
I20260811 19:11:58.287417 14846 raft_consensus.cc:374] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:11:58.287608 14846 raft_consensus.cc:729] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d719d69f5e494d2eaecba22f5241c8b4, State: Initialized, Role: FOLLOWER
I20260811 19:11:58.288409 14846 consensus_queue.cc:260] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } }
I20260811 19:11:58.288677 14846 raft_consensus.cc:388] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:11:58.288792 14846 raft_consensus.cc:482] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:11:58.288925 14846 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:11:58.291569 14846 raft_consensus.cc:504] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } }
I20260811 19:11:58.292234 14846 leader_election.cc:302] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d719d69f5e494d2eaecba22f5241c8b4; no voters:
I20260811 19:11:58.292919 14846 leader_election.cc:288] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:11:58.293078 14852 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:11:58.293830 14852 raft_consensus.cc:686] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4 [term 1 LEADER]: Becoming Leader. State: Replica: d719d69f5e494d2eaecba22f5241c8b4, State: Running, Role: LEADER
I20260811 19:11:58.294430 14852 consensus_queue.cc:237] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } }
I20260811 19:11:58.295318 14846 sys_catalog.cc:564] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:11:58.297415 14854 sys_catalog.cc:455] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d719d69f5e494d2eaecba22f5241c8b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } }
I20260811 19:11:58.298142 14855 sys_catalog.cc:455] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d719d69f5e494d2eaecba22f5241c8b4. Latest consensus state: current_term: 1 leader_uuid: "d719d69f5e494d2eaecba22f5241c8b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } }
I20260811 19:11:58.298844 14855 sys_catalog.cc:458] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4 [sys.catalog]: This master's current role is: LEADER
I20260811 19:11:58.298543 14854 sys_catalog.cc:458] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4 [sys.catalog]: This master's current role is: LEADER
I20260811 19:11:58.304328 14859 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:11:58.311460 14859 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:11:58.326273 14859 catalog_manager.cc:1347] Generated new cluster ID: e24c3dbff427436d95a3345db6017754
I20260811 19:11:58.326421 14859 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:11:58.341424 14859 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:11:58.342844 14859 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:11:58.355007 14859 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4: Generated new TSK 0
I20260811 19:11:58.356395 14859 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.3.61.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.61.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.61.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/krb5kdc/oryx_127.3.61.129.keytab.
Entry for principal oryx/127.3.61.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/krb5kdc/oryx_127.3.61.129.keytab.
I20260811 19:11:58.446332 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.129:0
--local_ip_for_outbound_sockets=127.3.61.129
--webserver_interface=127.3.61.129
--webserver_port=0
--tserver_master_addrs=127.3.61.190:33527
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/krb5kdc/oryx_127.3.61.129.keytab
--principal=oryx/127.3.61.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:34789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:11:58.874089 14880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:11:58.874413 14880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:11:58.874532 14880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:11:58.882288 14880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:11:58.882428 14880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:11:58.882586 14880 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.129
I20260811 19:11:58.893033 14880 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:34789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/krb5kdc/oryx_127.3.61.129.keytab
--principal=oryx/127.3.61.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.61.129
--webserver_port=0
--tserver_master_addrs=127.3.61.190:33527
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.129
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:11:58.895330 14880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:11:58.897608 14880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:11:58.912608 14887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:58.912519 14894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:58.912546 14889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:11:58.915310 14880 server_base.cc:1034] running on GCE node
I20260811 19:11:58.916541 14880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:11:58.918035 14880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:11:58.919351 14880 hybrid_clock.cc:648] HybridClock initialized: now 1786475518919269 us; error 80 us; skew 500 ppm
I20260811 19:11:58.919819 14880 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:11:58 dist-test-slave-t7g5 krb5kdc[14712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475518, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.61.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:11:58.923902 14880 init.cc:377] Logged in from keytab as oryx/127.3.61.129@KRBTEST.COM (short username oryx)
I20260811 19:11:58.925829 14880 webserver.cc:460] Webserver started at http://127.3.61.129:37447/ using document root <none> and password file <none>
I20260811 19:11:58.926656 14880 fs_manager.cc:359] Metadata directory not provided
I20260811 19:11:58.926774 14880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:11:58.927069 14880 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:11:58.930617 14880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-0/data/instance:
uuid: "3e21de71b6e9439f80d484412bed0c8a"
format_stamp: "Formatted at 2026-08-11 19:11:58 on dist-test-slave-t7g5"
server_key: "c0229b07bdc403d36a8fdf8326d40940"
server_key_iv: "ed6c7dbb86fd0756613af4007f7ea4b9"
server_key_version: "encryptionkey@0"
I20260811 19:11:58.931464 14880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-0/wal/instance:
uuid: "3e21de71b6e9439f80d484412bed0c8a"
format_stamp: "Formatted at 2026-08-11 19:11:58 on dist-test-slave-t7g5"
server_key: "c0229b07bdc403d36a8fdf8326d40940"
server_key_iv: "ed6c7dbb86fd0756613af4007f7ea4b9"
server_key_version: "encryptionkey@0"
I20260811 19:11:58.938328 14880 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260811 19:11:58.943224 14914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:58.945189 14880 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 19:11:58.945421 14880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-0/wal
uuid: "3e21de71b6e9439f80d484412bed0c8a"
format_stamp: "Formatted at 2026-08-11 19:11:58 on dist-test-slave-t7g5"
server_key: "c0229b07bdc403d36a8fdf8326d40940"
server_key_iv: "ed6c7dbb86fd0756613af4007f7ea4b9"
server_key_version: "encryptionkey@0"
I20260811 19:11:58.945779 14880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:11:58.968828 14880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:11:58.973605 14880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:11:58.973943 14880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:11:58.975068 14880 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:11:58.977243 14880 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:11:58.977339 14880 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:58.977464 14880 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:11:58.977535 14880 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:59.015551 14880 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.129:38449
I20260811 19:11:59.015813 15091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.129:38449 every 8 connection(s)
I20260811 19:11:59.017009 14880 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
I20260811 19:11:59.021318 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 14880
I20260811 19:11:59.021571 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-0/wal/instance
I20260811 19:11:59.022563 3318 external_mini_cluster.cc:1428] Setting key ea08b12d97ee29f940a5f5a90cfe236a
Aug 11 19:11:59 dist-test-slave-t7g5 krb5kdc[14712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475518, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.61.129@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM
I20260811 19:11:59.042677 15095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:59.020021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:47163 (local address 127.3.61.190:33527)
0811 19:11:59.020530 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:59.020534 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:59.021954 (+ 1420us) server_negotiation.cc:407] Connection header received
0811 19:11:59.024490 (+ 2536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:59.024502 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:59.024610 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:59.024768 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:59.027253 (+ 2485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:59.028142 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:59.030615 (+ 2473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:59.031059 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:59.031190 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:59.036032 (+ 4842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:59.036039 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:59.036042 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:59.036084 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:59.038720 (+ 2636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:59.039703 (+ 983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:59.039709 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:59.039712 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:59.039875 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:59.040501 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:59.040506 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:59.040508 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:59.040857 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:59.041016 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:59.041984 (+ 968us) server_negotiation.cc:299] Negotiation successful
0811 19:11:59.042256 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":263,"threads_started":1}
I20260811 19:11:59.043814 15096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:59.020535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33527 (local address 127.3.61.129:47163)
0811 19:11:59.021699 (+ 1164us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:59.021774 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:59.024117 (+ 2343us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:59.025114 (+ 997us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:59.025128 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:59.026000 (+ 872us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:11:59.026954 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:59.026993 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:59.028363 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:59.028369 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:59.030398 (+ 2029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:59.030418 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:59.031354 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:59.031363 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:59.031683 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:59.032634 (+ 951us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:11:59.032666 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:11:59.035680 (+ 3014us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:11:59.038930 (+ 3250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:59.038939 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:59.038956 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:59.039483 (+ 527us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:59.040048 (+ 565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:59.040072 (+ 24us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:59.040074 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:59.040334 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:59.041010 (+ 676us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:11:59.041020 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:11:59.041489 (+ 469us) client_negotiation.cc:769] Sending connection context
0811 19:11:59.041837 (+ 348us) client_negotiation.cc:240] Negotiation successful
0811 19:11:59.042467 (+ 630us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":772,"thread_start_us":523,"threads_started":1}
I20260811 19:11:59.045962 15092 heartbeater.cc:347] Connected to a master server at 127.3.61.190:33527
I20260811 19:11:59.046554 15092 heartbeater.cc:464] Registering TS with master...
I20260811 19:11:59.047909 15092 heartbeater.cc:511] Master 127.3.61.190:33527 requested a full tablet report, sending...
I20260811 19:11:59.050724 14792 ts_manager.cc:194] Registered new tserver with Master: 3e21de71b6e9439f80d484412bed0c8a (127.3.61.129:38449)
I20260811 19:11:59.052802 14792 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.61.129@KRBTEST.COM'} at 127.3.61.129:47163
WARNING: no policy specified for oryx/127.3.61.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.61.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.61.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/krb5kdc/oryx_127.3.61.130.keytab.
Entry for principal oryx/127.3.61.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/krb5kdc/oryx_127.3.61.130.keytab.
I20260811 19:11:59.093617 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.130:0
--local_ip_for_outbound_sockets=127.3.61.130
--webserver_interface=127.3.61.130
--webserver_port=0
--tserver_master_addrs=127.3.61.190:33527
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/krb5kdc/oryx_127.3.61.130.keytab
--principal=oryx/127.3.61.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:34789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:11:59.487186 15100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:11:59.487534 15100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:11:59.487663 15100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:11:59.495339 15100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:11:59.495515 15100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:11:59.495726 15100 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.130
I20260811 19:11:59.508780 15100 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:34789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/krb5kdc/oryx_127.3.61.130.keytab
--principal=oryx/127.3.61.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.61.130
--webserver_port=0
--tserver_master_addrs=127.3.61.190:33527
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.130
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:11:59.510264 15100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:11:59.511638 15100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:11:59.526618 15114 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:59.526594 15111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:11:59.526453 15107 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:11:59.528923 15100 server_base.cc:1034] running on GCE node
I20260811 19:11:59.529693 15100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:11:59.530793 15100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:11:59.532037 15100 hybrid_clock.cc:648] HybridClock initialized: now 1786475519531993 us; error 43 us; skew 500 ppm
I20260811 19:11:59.532401 15100 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:11:59 dist-test-slave-t7g5 krb5kdc[14712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475519, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.61.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:11:59.535969 15100 init.cc:377] Logged in from keytab as oryx/127.3.61.130@KRBTEST.COM (short username oryx)
I20260811 19:11:59.537881 15100 webserver.cc:460] Webserver started at http://127.3.61.130:46603/ using document root <none> and password file <none>
I20260811 19:11:59.538691 15100 fs_manager.cc:359] Metadata directory not provided
I20260811 19:11:59.538820 15100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:11:59.539235 15100 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:11:59.543146 15100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-1/data/instance:
uuid: "06c375ae44c0466c8b91db22d0b0765c"
format_stamp: "Formatted at 2026-08-11 19:11:59 on dist-test-slave-t7g5"
server_key: "a383bed5a9214effd78e77ef03c84f85"
server_key_iv: "efd1b422c5b65fdf6b413f6fc1beece2"
server_key_version: "encryptionkey@0"
I20260811 19:11:59.544456 15100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-1/wal/instance:
uuid: "06c375ae44c0466c8b91db22d0b0765c"
format_stamp: "Formatted at 2026-08-11 19:11:59 on dist-test-slave-t7g5"
server_key: "a383bed5a9214effd78e77ef03c84f85"
server_key_iv: "efd1b422c5b65fdf6b413f6fc1beece2"
server_key_version: "encryptionkey@0"
I20260811 19:11:59.551810 15100 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.001s sys 0.007s
I20260811 19:11:59.556327 15124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:59.558640 15100 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260811 19:11:59.558931 15100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-1/wal
uuid: "06c375ae44c0466c8b91db22d0b0765c"
format_stamp: "Formatted at 2026-08-11 19:11:59 on dist-test-slave-t7g5"
server_key: "a383bed5a9214effd78e77ef03c84f85"
server_key_iv: "efd1b422c5b65fdf6b413f6fc1beece2"
server_key_version: "encryptionkey@0"
I20260811 19:11:59.559202 15100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:11:59.589128 15100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:11:59.593883 15100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:11:59.594202 15100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:11:59.595434 15100 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:11:59.597117 15100 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:11:59.597200 15100 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:59.597326 15100 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:11:59.597381 15100 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:11:59.635995 15100 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.130:32931
I20260811 19:11:59.636083 15300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.130:32931 every 8 connection(s)
I20260811 19:11:59.637383 15100 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
I20260811 19:11:59.638441 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 15100
I20260811 19:11:59.638636 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-1/wal/instance
I20260811 19:11:59.639156 3318 external_mini_cluster.cc:1428] Setting key 89a994ff830b64d5fda45dc529e265af
Aug 11 19:11:59 dist-test-slave-t7g5 krb5kdc[14712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475519, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.61.130@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM
I20260811 19:11:59.663542 15303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:59.641115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:43759 (local address 127.3.61.190:33527)
0811 19:11:59.641736 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0811 19:11:59.641744 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:11:59.643094 (+ 1350us) server_negotiation.cc:407] Connection header received
0811 19:11:59.645810 (+ 2716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:11:59.645818 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:11:59.645930 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:11:59.646048 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:11:59.648027 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:59.648818 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:59.650950 (+ 2132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:11:59.651438 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:11:59.651568 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:59.655822 (+ 4254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:11:59.655832 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:11:59.655837 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:11:59.655890 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:11:59.659674 (+ 3784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:59.660772 (+ 1098us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:59.660778 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:59.660782 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:59.660946 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:11:59.661507 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:11:59.661512 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:11:59.661515 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:11:59.661934 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:11:59.662095 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 19:11:59.662909 (+ 814us) server_negotiation.cc:299] Negotiation successful
0811 19:11:59.663178 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":280,"threads_started":1}
I20260811 19:11:59.664563 15304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:11:59.641776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33527 (local address 127.3.61.130:43759)
0811 19:11:59.642868 (+ 1092us) negotiation.cc:107] Waiting for socket to connect
0811 19:11:59.642925 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 19:11:59.645333 (+ 2408us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:11:59.646458 (+ 1125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:11:59.646469 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:11:59.647102 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:11:59.647777 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:59.647804 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:59.649009 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:59.649013 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:59.650662 (+ 1649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:11:59.650686 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:11:59.651708 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:11:59.651716 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:11:59.651967 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:11:59.652692 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:11:59.652718 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:11:59.655403 (+ 2685us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:11:59.660029 (+ 4626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:59.660040 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:59.660058 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:59.660605 (+ 547us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:59.661104 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:11:59.661109 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:11:59.661111 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:11:59.661366 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:11:59.662108 (+ 742us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:11:59.662115 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:11:59.662496 (+ 381us) client_negotiation.cc:769] Sending connection context
0811 19:11:59.662762 (+ 266us) client_negotiation.cc:240] Negotiation successful
0811 19:11:59.663263 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":727,"thread_start_us":425,"threads_started":1}
I20260811 19:11:59.666393 15301 heartbeater.cc:347] Connected to a master server at 127.3.61.190:33527
I20260811 19:11:59.666788 15301 heartbeater.cc:464] Registering TS with master...
I20260811 19:11:59.667771 15301 heartbeater.cc:511] Master 127.3.61.190:33527 requested a full tablet report, sending...
I20260811 19:11:59.670235 14792 ts_manager.cc:194] Registered new tserver with Master: 06c375ae44c0466c8b91db22d0b0765c (127.3.61.130:32931)
I20260811 19:11:59.671490 14792 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.61.130@KRBTEST.COM'} at 127.3.61.130:43759
WARNING: no policy specified for oryx/127.3.61.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.61.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.61.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/krb5kdc/oryx_127.3.61.131.keytab.
Entry for principal oryx/127.3.61.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/krb5kdc/oryx_127.3.61.131.keytab.
I20260811 19:11:59.713955 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.131:0
--local_ip_for_outbound_sockets=127.3.61.131
--webserver_interface=127.3.61.131
--webserver_port=0
--tserver_master_addrs=127.3.61.190:33527
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/krb5kdc/oryx_127.3.61.131.keytab
--principal=oryx/127.3.61.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:34789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:12:00.057525 15092 heartbeater.cc:503] Master 127.3.61.190:33527 was elected leader, sending a full tablet report...
W20260811 19:12:00.113703 15308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:12:00.114063 15308 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:12:00.114174 15308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:12:00.122910 15308 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:12:00.123056 15308 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:12:00.123178 15308 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.131
I20260811 19:12:00.133757 15308 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:34789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/krb5kdc/oryx_127.3.61.131.keytab
--principal=oryx/127.3.61.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.61.131
--webserver_port=0
--tserver_master_addrs=127.3.61.190:33527
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.131
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:12:00.135160 15308 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:12:00.136591 15308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:12:00.153546 15321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:00.153581 15315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:00.153512 15316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:12:00.154356 15308 server_base.cc:1034] running on GCE node
I20260811 19:12:00.155064 15308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:12:00.156188 15308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:12:00.157455 15308 hybrid_clock.cc:648] HybridClock initialized: now 1786475520157405 us; error 38 us; skew 500 ppm
I20260811 19:12:00.157882 15308 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:12:00 dist-test-slave-t7g5 krb5kdc[14712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475520, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.61.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:12:00.161258 15308 init.cc:377] Logged in from keytab as oryx/127.3.61.131@KRBTEST.COM (short username oryx)
I20260811 19:12:00.163096 15308 webserver.cc:460] Webserver started at http://127.3.61.131:33447/ using document root <none> and password file <none>
I20260811 19:12:00.163894 15308 fs_manager.cc:359] Metadata directory not provided
I20260811 19:12:00.164058 15308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:12:00.164403 15308 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:12:00.167718 15308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-2/data/instance:
uuid: "9117d1dafdbe4e6f804a6f7f742596b6"
format_stamp: "Formatted at 2026-08-11 19:12:00 on dist-test-slave-t7g5"
server_key: "4678e81847ce83504b3bf729f70388c5"
server_key_iv: "271ae640c3f2b54d970a228a6a2d37e3"
server_key_version: "encryptionkey@0"
I20260811 19:12:00.168565 15308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-2/wal/instance:
uuid: "9117d1dafdbe4e6f804a6f7f742596b6"
format_stamp: "Formatted at 2026-08-11 19:12:00 on dist-test-slave-t7g5"
server_key: "4678e81847ce83504b3bf729f70388c5"
server_key_iv: "271ae640c3f2b54d970a228a6a2d37e3"
server_key_version: "encryptionkey@0"
I20260811 19:12:00.174710 15308 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260811 19:12:00.179107 15334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:00.181465 15308 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.002s
I20260811 19:12:00.181809 15308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-2/wal
uuid: "9117d1dafdbe4e6f804a6f7f742596b6"
format_stamp: "Formatted at 2026-08-11 19:12:00 on dist-test-slave-t7g5"
server_key: "4678e81847ce83504b3bf729f70388c5"
server_key_iv: "271ae640c3f2b54d970a228a6a2d37e3"
server_key_version: "encryptionkey@0"
I20260811 19:12:00.182092 15308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:12:00.201017 15308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:12:00.207688 15308 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:12:00.208110 15308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:12:00.209632 15308 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:12:00.211477 15308 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:12:00.211580 15308 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:00.211722 15308 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:12:00.211779 15308 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:00.251533 15308 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.131:34125
I20260811 19:12:00.251579 15509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.131:34125 every 8 connection(s)
I20260811 19:12:00.252928 15308 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
I20260811 19:12:00.259670 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 15308
I20260811 19:12:00.259922 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-2/wal/instance
I20260811 19:12:00.260517 3318 external_mini_cluster.cc:1428] Setting key 6c52c2326de4a97a6111dd03dd29a2ef
Aug 11 19:12:00 dist-test-slave-t7g5 krb5kdc[14712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475520, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.61.131@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM
I20260811 19:12:00.277174 15514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:00.255936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:40159 (local address 127.3.61.190:33527)
0811 19:12:00.256439 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:00.256444 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:00.257474 (+ 1030us) server_negotiation.cc:407] Connection header received
0811 19:12:00.259511 (+ 2037us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:00.259517 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:00.259605 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:00.259703 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:00.261588 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:00.262660 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:00.264917 (+ 2257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:00.265417 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:00.265536 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:00.270281 (+ 4745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:00.270287 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:00.270291 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:00.270331 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:00.273309 (+ 2978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:00.274334 (+ 1025us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:00.274339 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:00.274341 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:00.274506 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:00.275145 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:00.275151 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:00.275154 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:00.275535 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:00.275695 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:00.276524 (+ 829us) server_negotiation.cc:299] Negotiation successful
0811 19:12:00.276776 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":257,"threads_started":1}
I20260811 19:12:00.278337 15515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:00.256393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33527 (local address 127.3.61.131:40159)
0811 19:12:00.257292 (+ 899us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:00.257340 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:00.259198 (+ 1858us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:00.259999 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:00.260011 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:00.260640 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:00.261389 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:00.261417 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:00.262858 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:00.262864 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:00.264694 (+ 1830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:00.264723 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:00.265786 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:00.265795 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:00.266111 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:00.266996 (+ 885us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:00.267028 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:00.269961 (+ 2933us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:12:00.273537 (+ 3576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:00.273546 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:00.273571 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:00.274194 (+ 623us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:00.274672 (+ 478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:00.274677 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:00.274680 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:00.274981 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:00.275687 (+ 706us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:12:00.275697 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:12:00.276113 (+ 416us) client_negotiation.cc:769] Sending connection context
0811 19:12:00.276378 (+ 265us) client_negotiation.cc:240] Negotiation successful
0811 19:12:00.277009 (+ 631us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"thread_start_us":366,"threads_started":1}
I20260811 19:12:00.280382 15510 heartbeater.cc:347] Connected to a master server at 127.3.61.190:33527
I20260811 19:12:00.280944 15510 heartbeater.cc:464] Registering TS with master...
I20260811 19:12:00.282068 15510 heartbeater.cc:511] Master 127.3.61.190:33527 requested a full tablet report, sending...
I20260811 19:12:00.284189 14792 ts_manager.cc:194] Registered new tserver with Master: 9117d1dafdbe4e6f804a6f7f742596b6 (127.3.61.131:34125)
I20260811 19:12:00.285065 14792 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.61.131@KRBTEST.COM'} at 127.3.61.131:40159
I20260811 19:12:00.290182 3318 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.3.61.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.61.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.61.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/krb5kdc/oryx_127.3.61.189.keytab.
Entry for principal oryx/127.3.61.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/krb5kdc/oryx_127.3.61.189.keytab.
I20260811 19:12:00.357259 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.189:43587
--webserver_interface=127.3.61.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/krb5kdc/oryx_127.3.61.189.keytab
--principal=oryx/127.3.61.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:34789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.61.190:33527,127.3.61.189:43587
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:12:00.675627 15301 heartbeater.cc:503] Master 127.3.61.190:33527 was elected leader, sending a full tablet report...
W20260811 19:12:00.736933 15519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:12:00.737284 15519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:12:00.737437 15519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:12:00.746402 15519 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:12:00.746585 15519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:12:00.746672 15519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:12:00.746754 15519 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:12:00.746819 15519 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:12:00.759033 15519 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:34789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.3.61.190:33527,127.3.61.189:43587
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/krb5kdc/oryx_127.3.61.189.keytab
--principal=oryx/127.3.61.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.189:43587
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.3.61.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
W20260811 19:12:00.759464 15519 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.3.61.190:33527,127.3.61.189:43587'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260811 19:12:00.760704 15519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:12:00.762187 15519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:12:00.775398 15533 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:00.775370 15536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:00.775465 15534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:12:00.781304 15519 server_base.cc:1034] running on GCE node
I20260811 19:12:00.782359 15519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:12:00.784034 15519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:12:00.785341 15519 hybrid_clock.cc:648] HybridClock initialized: now 1786475520785252 us; error 72 us; skew 500 ppm
I20260811 19:12:00.785799 15519 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:12:00 dist-test-slave-t7g5 krb5kdc[14712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475520, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.61.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:12:00.789680 15519 init.cc:377] Logged in from keytab as oryx/127.3.61.189@KRBTEST.COM (short username oryx)
I20260811 19:12:00.791587 15519 webserver.cc:460] Webserver started at http://127.3.61.189:41179/ using document root <none> and password file <none>
I20260811 19:12:00.792428 15519 fs_manager.cc:359] Metadata directory not provided
I20260811 19:12:00.792529 15519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:12:00.792850 15519 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:12:00.796152 15519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-1/data/instance:
uuid: "0c62dbd92db642fa9b56acc74117c8c0"
format_stamp: "Formatted at 2026-08-11 19:12:00 on dist-test-slave-t7g5"
server_key: "8745c31fe5e55daf3b14f7cef8bb1045"
server_key_iv: "5db41c42668c230b6079c299aee7f50c"
server_key_version: "encryptionkey@0"
I20260811 19:12:00.797223 15519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-1/wal/instance:
uuid: "0c62dbd92db642fa9b56acc74117c8c0"
format_stamp: "Formatted at 2026-08-11 19:12:00 on dist-test-slave-t7g5"
server_key: "8745c31fe5e55daf3b14f7cef8bb1045"
server_key_iv: "5db41c42668c230b6079c299aee7f50c"
server_key_version: "encryptionkey@0"
I20260811 19:12:00.802984 15519 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 19:12:00.806713 15545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:00.808555 15519 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 19:12:00.808734 15519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-1/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-1/wal
uuid: "0c62dbd92db642fa9b56acc74117c8c0"
format_stamp: "Formatted at 2026-08-11 19:12:00 on dist-test-slave-t7g5"
server_key: "8745c31fe5e55daf3b14f7cef8bb1045"
server_key_iv: "5db41c42668c230b6079c299aee7f50c"
server_key_version: "encryptionkey@0"
I20260811 19:12:00.808975 15519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:12:00.827150 15519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:12:00.832257 15519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:12:00.832581 15519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:12:00.853499 15519 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.189:43587
I20260811 19:12:00.853516 15625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.189:43587 every 8 connection(s)
I20260811 19:12:00.855129 15519 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-1/data/info.pb
I20260811 19:12:00.858626 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 15519
I20260811 19:12:00.860845 15627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:12:00.870896 15627 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 19:12:00 dist-test-slave-t7g5 krb5kdc[14712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475517, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.61.189@KRBTEST.COM
Aug 11 19:12:00 dist-test-slave-t7g5 krb5kdc[14712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475520, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.61.189@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM
I20260811 19:12:00.888201 15635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:00.861089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41798 (local address 127.3.61.189:43587)
0811 19:12:00.861974 (+ 885us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:00.861990 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:00.862043 (+ 53us) server_negotiation.cc:407] Connection header received
0811 19:12:00.864299 (+ 2256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:00.864331 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:00.864890 (+ 559us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:00.865392 (+ 502us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:00.866945 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:00.869011 (+ 2066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:00.870539 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:00.871625 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:00.871828 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:00.878495 (+ 6667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:00.878505 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:00.878517 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:00.878559 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:00.882799 (+ 4240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:00.883672 (+ 873us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:00.883685 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:00.883695 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:00.883929 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:00.884470 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:00.884474 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:00.884476 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:00.885231 (+ 755us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:00.885570 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:00.886304 (+ 734us) server_negotiation.cc:299] Negotiation successful
0811 19:12:00.887037 (+ 733us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":642,"thread_start_us":353,"threads_started":1}
I20260811 19:12:00.893002 15636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:00.874099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58358 (local address 127.3.61.190:33527)
0811 19:12:00.874806 (+ 707us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:00.874814 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:00.875307 (+ 493us) server_negotiation.cc:407] Connection header received
0811 19:12:00.875522 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:00.875528 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:00.875782 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:00.875934 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:00.877860 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:00.878806 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:00.881239 (+ 2433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:00.881703 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:00.881899 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:00.885785 (+ 3886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:00.885793 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:00.885798 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:00.885864 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:00.889990 (+ 4126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:00.890862 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:00.890868 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:00.890871 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:00.891047 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:00.891470 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:00.891475 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:00.891477 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:00.891844 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:00.891976 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:00.892521 (+ 545us) server_negotiation.cc:299] Negotiation successful
0811 19:12:00.892721 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":511,"thread_start_us":308,"threads_started":1}
I20260811 19:12:00.893229 15637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:00.874034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33527 (local address 127.0.0.1:58358)
0811 19:12:00.875115 (+ 1081us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:00.875168 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:00.875349 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:00.876159 (+ 810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:00.876169 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:00.877332 (+ 1163us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:00.877637 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:00.877678 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:00.878961 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:00.878966 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:00.880894 (+ 1928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:00.880927 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:00.881923 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:00.881931 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:00.882136 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:00.882799 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:00.882823 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:00.885504 (+ 2681us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:12:00.890189 (+ 4685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:00.890202 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:00.890209 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:00.890710 (+ 501us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:00.891162 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:00.891167 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:00.891171 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:00.891314 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:00.891996 (+ 682us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:12:00.892003 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:12:00.892177 (+ 174us) client_negotiation.cc:769] Sending connection context
0811 19:12:00.892361 (+ 184us) client_negotiation.cc:240] Negotiation successful
0811 19:12:00.892889 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":824,"thread_start_us":643,"threads_started":1}
I20260811 19:12:00.895803 15627 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43587 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 19:12:00 dist-test-slave-t7g5 krb5kdc[14712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475520, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.61.189@KRBTEST.COM for oryx/127.3.61.189@KRBTEST.COM
I20260811 19:12:00.910095 15637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:00.896688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.189:43587 (local address 127.0.0.1:41810)
0811 19:12:00.896907 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:00.896928 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:00.897031 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:00.897329 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:00.897332 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:00.897750 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:00.897905 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:00.897913 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:00.898940 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:00.898944 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:00.899884 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:00.899897 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:00.900960 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:00.900966 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:00.901146 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:00.902117 (+ 971us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:00.902144 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:00.904724 (+ 2580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:12:00.907713 (+ 2989us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:00.907719 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:00.907723 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:00.908115 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:00.908499 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:00.908504 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:00.908507 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:00.908634 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:00.909122 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:12:00.909127 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:12:00.909255 (+ 128us) client_negotiation.cc:769] Sending connection context
0811 19:12:00.909381 (+ 126us) client_negotiation.cc:240] Negotiation successful
0811 19:12:00.909715 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260811 19:12:00.910244 15635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:00.896958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41810 (local address 127.3.61.189:43587)
0811 19:12:00.897104 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:00.897108 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:00.897124 (+ 16us) server_negotiation.cc:407] Connection header received
0811 19:12:00.897167 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:00.897170 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:00.897226 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:00.897314 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:00.898057 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:00.898791 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:00.900047 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:00.900708 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:00.900885 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:00.904936 (+ 4051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:00.904941 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:00.904945 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:00.904979 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:00.907573 (+ 2594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:00.908228 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:00.908233 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:00.908236 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:00.908377 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:00.908734 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:00.908737 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:00.908739 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:00.909012 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:00.909141 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:00.909499 (+ 358us) server_negotiation.cc:299] Negotiation successful
0811 19:12:00.909874 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":138,"server-negotiator.queue_time_us":42}
I20260811 19:12:00.916049 15627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0: Bootstrap starting.
I20260811 19:12:00.919482 15627 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0: Neither blocks nor log segments found. Creating new log.
I20260811 19:12:00.920584 15627 log.cc:824] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0: Log is configured to *not* fsync() on all Append() calls
I20260811 19:12:00.923955 15627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0: No bootstrap required, opened a new log
I20260811 19:12:00.931340 15627 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43587 } }
I20260811 19:12:00.931701 15627 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:12:00.931816 15627 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c62dbd92db642fa9b56acc74117c8c0, State: Initialized, Role: FOLLOWER
I20260811 19:12:00.932446 15627 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43587 } }
I20260811 19:12:00.933761 15639 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43587 } } }
I20260811 19:12:00.934257 15639 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:12:00.934571 15627 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 19:12:00.944470 15659 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 19:12:00.944744 15659 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 19:12:00.949184 15519 master_runner.cc:428] Detected that this master 0c62dbd92db642fa9b56acc74117c8c0 is joining an existing cluster
I20260811 19:12:00.949339 15519 master_runner.cc:432] Initiating AddMaster RPC to add 127.3.61.189:43587
I20260811 19:12:00.978507 15636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:00.955972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58368 (local address 127.3.61.190:33527)
0811 19:12:00.956184 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:00.956191 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:00.960132 (+ 3941us) server_negotiation.cc:407] Connection header received
0811 19:12:00.960340 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:00.960345 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:00.960439 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:00.960534 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:00.962147 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:00.963178 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:00.965233 (+ 2055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:00.965873 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:00.965968 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:00.970226 (+ 4258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:00.970234 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:00.970239 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:00.970286 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:00.974867 (+ 4581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:00.975700 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:00.975705 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:00.975708 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:00.975886 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:00.976375 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:00.976380 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:00.976383 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:00.977079 (+ 696us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:00.977236 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:00.977901 (+ 665us) server_negotiation.cc:299] Negotiation successful
0811 19:12:00.978176 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 19:12:00.980341 15667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:00.956545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33527 (local address 127.0.0.1:58368)
0811 19:12:00.959981 (+ 3436us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:00.960011 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:00.960193 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:00.960720 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:00.960724 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:00.961457 (+ 733us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:00.961955 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:00.961970 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:00.963379 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:00.963384 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:00.964918 (+ 1534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:00.965015 (+ 97us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:00.966277 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:00.966283 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:00.966458 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:00.967507 (+ 1049us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:00.967631 (+ 124us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:00.969962 (+ 2331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:12:00.975045 (+ 5083us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:00.975050 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:00.975054 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:00.975553 (+ 499us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:00.976035 (+ 482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:00.976040 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:00.976042 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:00.976221 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:00.977329 (+ 1108us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:12:00.977333 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:12:00.977559 (+ 226us) client_negotiation.cc:769] Sending connection context
0811 19:12:00.979184 (+ 1625us) client_negotiation.cc:240] Negotiation successful
0811 19:12:00.979597 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3285,"thread_start_us":7139,"threads_started":1}
I20260811 19:12:00.980767 15635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:00.956040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41812 (local address 127.3.61.189:43587)
0811 19:12:00.956262 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:00.956267 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:00.963665 (+ 7398us) server_negotiation.cc:407] Connection header received
0811 19:12:00.964131 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:00.964136 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:00.964203 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:00.964265 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:00.965847 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:00.966926 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:00.969054 (+ 2128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:00.969810 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:00.969914 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:00.972463 (+ 2549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:00.972476 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:00.972481 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:00.972518 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:00.976680 (+ 4162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:00.977794 (+ 1114us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:00.977801 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:00.977804 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:00.978006 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:00.978469 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:00.978474 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:00.978477 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:00.979036 (+ 559us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:00.979223 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:00.979788 (+ 565us) server_negotiation.cc:299] Negotiation successful
0811 19:12:00.980104 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 19:12:00.982401 15666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:00.956582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.189:43587 (local address 127.0.0.1:41812)
0811 19:12:00.963504 (+ 6922us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:00.963525 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:00.963981 (+ 456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:00.964500 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:00.964504 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:00.965337 (+ 833us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:00.965671 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:00.965708 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:00.967143 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:00.967148 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:00.968846 (+ 1698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:00.968864 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:00.970317 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:00.970323 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:00.970495 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:00.971148 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:00.971172 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:00.972250 (+ 1078us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:12:00.976925 (+ 4675us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:00.976930 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:00.976935 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:00.977628 (+ 693us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:00.978169 (+ 541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:00.978174 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:00.978178 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:00.978347 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:00.979325 (+ 978us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:12:00.979329 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:12:00.979509 (+ 180us) client_negotiation.cc:769] Sending connection context
0811 19:12:00.981571 (+ 2062us) client_negotiation.cc:240] Negotiation successful
0811 19:12:00.981955 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6760,"thread_start_us":3094,"threads_started":1}
WARNING: no policy specified for oryx/127.3.61.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.61.188@KRBTEST.COM" created.
Aug 11 19:12:00 dist-test-slave-t7g5 krb5kdc[14712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475518, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.61.190@KRBTEST.COM for oryx/127.3.61.189@KRBTEST.COM
I20260811 19:12:01.008558 15670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:00.990077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.189:43587 (local address 127.0.0.1:41822)
0811 19:12:00.990702 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:00.990749 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:00.990944 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:00.991513 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:00.991520 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:00.992204 (+ 684us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:00.992432 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:00.992443 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:00.993526 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:00.993530 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:00.994724 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:00.994739 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:00.995633 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:00.995639 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:00.995846 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:00.996822 (+ 976us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:00.996851 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:00.999377 (+ 2526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:12:01.003979 (+ 4602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:01.004003 (+ 24us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:01.004012 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:01.005114 (+ 1102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:01.005824 (+ 710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:01.005830 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:01.005833 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:01.006097 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:01.007006 (+ 909us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:12:01.007017 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:12:01.007250 (+ 233us) client_negotiation.cc:769] Sending connection context
0811 19:12:01.007491 (+ 241us) client_negotiation.cc:240] Negotiation successful
0811 19:12:01.008066 (+ 575us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":305,"threads_started":1}
I20260811 19:12:01.008920 15635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:00.990334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41822 (local address 127.3.61.189:43587)
0811 19:12:00.991161 (+ 827us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:00.991167 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:00.991181 (+ 14us) server_negotiation.cc:407] Connection header received
0811 19:12:00.991252 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:00.991255 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:00.991342 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:00.991434 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:00.992601 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:00.993380 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:00.994884 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:00.995448 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:00.995582 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:00.999591 (+ 4009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:00.999597 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:00.999600 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:00.999634 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:01.003726 (+ 4092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:01.005335 (+ 1609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:01.005343 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:01.005347 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:01.005594 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:01.006236 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:01.006242 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:01.006245 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:01.006732 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:01.006903 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:01.008064 (+ 1161us) server_negotiation.cc:299] Negotiation successful
0811 19:12:01.008330 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":705}
I20260811 19:12:01.011101 14792 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.3.61.189:43587: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0"
member_type: NON_VOTER
last_known_addr {
host: "127.3.61.189"
port: 43587
}
attrs {
promote: true
}
}
dest_uuid: "d719d69f5e494d2eaecba22f5241c8b4"
cas_config_opid_index: -1
I20260811 19:12:01.012526 14792 consensus_queue.cc:237] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: NON_VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: true } }
I20260811 19:12:01.028352 15589 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:12:01.032598 15589 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0 [term 1 FOLLOWER]: Refusing update from remote peer d719d69f5e494d2eaecba22f5241c8b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260811 19:12:01.034137 15675 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: NON_VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:12:01.033634 15676 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 0c62dbd92db642fa9b56acc74117c8c0 (127.3.61.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: NON_VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: true } } }
I20260811 19:12:01.038056 15676 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.3.61.189:43587
I20260811 19:12:01.038735 15675 sys_catalog.cc:455] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d719d69f5e494d2eaecba22f5241c8b4" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: NON_VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: true } } }
I20260811 19:12:01.039073 15675 sys_catalog.cc:458] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4 [sys.catalog]: This master's current role is: LEADER
I20260811 19:12:01.039777 15519 master.cc:537] Master@127.3.61.189:43587 shutting down...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.61.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/krb5kdc/oryx_127.3.61.188.keytab.
Entry for principal oryx/127.3.61.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/krb5kdc/oryx_127.3.61.188.keytab.
I20260811 19:12:01.043340 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.188:46509
--webserver_interface=127.3.61.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/krb5kdc/oryx_127.3.61.188.keytab
--principal=oryx/127.3.61.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:34789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.61.190:33527,127.3.61.189:43587,127.3.61.188:46509
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:12:01.071884 15589 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0 [term 1 LEARNER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 0c62dbd92db642fa9b56acc74117c8c0 (127.3.61.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: NON_VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: true } } }
I20260811 19:12:01.082288 15639 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d719d69f5e494d2eaecba22f5241c8b4. Latest consensus state: current_term: 1 leader_uuid: "d719d69f5e494d2eaecba22f5241c8b4" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: NON_VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: true } } }
I20260811 19:12:01.082587 15639 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0 [sys.catalog]: This master's current role is: LEARNER
I20260811 19:12:01.083004 15697 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d719d69f5e494d2eaecba22f5241c8b4" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: NON_VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: true } } }
I20260811 19:12:01.083087 15691 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d719d69f5e494d2eaecba22f5241c8b4" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: NON_VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: true } } }
I20260811 19:12:01.083489 15691 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0 [sys.catalog]: This master's current role is: LEARNER
I20260811 19:12:01.083397 15697 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0 [sys.catalog]: This master's current role is: LEARNER
I20260811 19:12:01.087249 15519 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0 [term 1 LEARNER]: Raft consensus shutting down.
I20260811 19:12:01.087888 15519 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0 [term 1 LEARNER]: Raft consensus is shut down!
I20260811 19:12:01.088066 15519 tablet_replica.cc:330] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0: stopping tablet replica
I20260811 19:12:01.107997 15519 master.cc:552] Master@127.3.61.189:43587 shutdown complete.
I20260811 19:12:01.108253 15519 master_runner.cc:305] Clearing existing system tablet
I20260811 19:12:01.108778 15519 fs_manager.cc:359] Metadata directory not provided
I20260811 19:12:01.108892 15519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:12:01.112249 15519 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.001s
I20260811 19:12:01.114200 15700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:01.114641 15519 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:12:01.114931 15519 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 19:12:01.115043 15519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-1/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-1/wal
uuid: "0c62dbd92db642fa9b56acc74117c8c0"
format_stamp: "Formatted at 2026-08-11 19:12:00 on dist-test-slave-t7g5"
server_key: "8745c31fe5e55daf3b14f7cef8bb1045"
server_key_iv: "5db41c42668c230b6079c299aee7f50c"
server_key_version: "encryptionkey@0"
I20260811 19:12:01.120196 15519 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:12:01.128679 15519 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 19:12:01.128898 15519 log.cc:1196] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260811 19:12:01.129436 15519 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0: Deleting consensus metadata
I20260811 19:12:01.129741 15519 master_runner.cc:315] Copying system tablet from 127.3.61.190:33527
I20260811 19:12:01.139950 15519 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0: tablet copy: Beginning tablet copy session from remote peer at address 127.3.61.190:33527
I20260811 19:12:01.157094 15636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:01.141369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58378 (local address 127.3.61.190:33527)
0811 19:12:01.141552 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:01.141558 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:01.141679 (+ 121us) server_negotiation.cc:407] Connection header received
0811 19:12:01.142000 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:01.142006 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:01.142093 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:01.142205 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:01.143235 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:01.144327 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:01.146250 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:01.146939 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:01.147074 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:01.149870 (+ 2796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:01.149886 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:01.149892 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:01.149944 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:01.153858 (+ 3914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:01.154812 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:01.154818 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:01.154822 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:01.154986 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:01.155409 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:01.155414 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:01.155418 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:01.155817 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:01.155979 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:01.156473 (+ 494us) server_negotiation.cc:299] Negotiation successful
0811 19:12:01.156714 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 19:12:01.157167 15712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:01.140939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33527 (local address 127.0.0.1:58378)
0811 19:12:01.141528 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:01.141555 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:01.141819 (+ 264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:01.142250 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:01.142255 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:01.142811 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:01.143072 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:01.143085 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:01.144489 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:01.144495 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:01.146084 (+ 1589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:01.146101 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:01.147113 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:01.147119 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:01.147313 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:01.148288 (+ 975us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:01.148325 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:01.149451 (+ 1126us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:12:01.154048 (+ 4597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:01.154055 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:01.154059 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:01.154652 (+ 593us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:01.155112 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:01.155117 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:01.155120 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:01.155291 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:01.155964 (+ 673us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:12:01.155969 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:12:01.156179 (+ 210us) client_negotiation.cc:769] Sending connection context
0811 19:12:01.156348 (+ 169us) client_negotiation.cc:240] Negotiation successful
0811 19:12:01.156724 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":617,"threads_started":1}
I20260811 19:12:01.158449 14824 tablet_copy_service.cc:140] P d719d69f5e494d2eaecba22f5241c8b4: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 0c62dbd92db642fa9b56acc74117c8c0 ({username='oryx', principal='oryx/127.3.61.189@KRBTEST.COM'} at 127.0.0.1:58378)
I20260811 19:12:01.158804 14824 tablet_copy_service.cc:161] P d719d69f5e494d2eaecba22f5241c8b4: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 0c62dbd92db642fa9b56acc74117c8c0 at {username='oryx', principal='oryx/127.3.61.189@KRBTEST.COM'} at 127.0.0.1:58378: session id = 0c62dbd92db642fa9b56acc74117c8c0-00000000000000000000000000000000
I20260811 19:12:01.164659 14824 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 19:12:01.168179 15519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 19:12:01.175859 15519 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0: tablet copy: Starting download of 0 data blocks...
I20260811 19:12:01.176294 15519 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0: tablet copy: Starting download of 1 WAL segments...
I20260811 19:12:01.179893 15519 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 19:12:01.185757 14824 tablet_copy_service.cc:342] P d719d69f5e494d2eaecba22f5241c8b4: Request end of tablet copy session 0c62dbd92db642fa9b56acc74117c8c0-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.3.61.189@KRBTEST.COM'} at 127.0.0.1:58378
I20260811 19:12:01.186138 14824 tablet_copy_service.cc:433] P d719d69f5e494d2eaecba22f5241c8b4: ending tablet copy session 0c62dbd92db642fa9b56acc74117c8c0-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 0c62dbd92db642fa9b56acc74117c8c0
I20260811 19:12:01.203271 15519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:12:01.206439 15713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:01.211465 15714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:01.219091 15716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:12:01.219614 15519 server_base.cc:1034] running on GCE node
I20260811 19:12:01.220005 15519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:12:01.220980 15519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:12:01.222301 15519 hybrid_clock.cc:648] HybridClock initialized: now 1786475521222223 us; error 40 us; skew 500 ppm
I20260811 19:12:01.222644 15519 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:12:01 dist-test-slave-t7g5 krb5kdc[14712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475521, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.61.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:12:01.225075 15519 init.cc:377] Logged in from keytab as oryx/127.3.61.189@KRBTEST.COM (short username oryx)
I20260811 19:12:01.227006 15519 webserver.cc:460] Webserver started at http://127.3.61.189:34503/ using document root <none> and password file <none>
I20260811 19:12:01.227596 15519 fs_manager.cc:359] Metadata directory not provided
I20260811 19:12:01.227803 15519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:12:01.231316 15519 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260811 19:12:01.233572 15736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:01.234103 15519 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 19:12:01.234289 15519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-1/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-1/wal
uuid: "0c62dbd92db642fa9b56acc74117c8c0"
format_stamp: "Formatted at 2026-08-11 19:12:00 on dist-test-slave-t7g5"
server_key: "8745c31fe5e55daf3b14f7cef8bb1045"
server_key_iv: "5db41c42668c230b6079c299aee7f50c"
server_key_version: "encryptionkey@0"
I20260811 19:12:01.234568 15519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:12:01.273289 15519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:12:01.278131 15519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:12:01.289004 15510 heartbeater.cc:503] Master 127.3.61.190:33527 was elected leader, sending a full tablet report...
I20260811 19:12:01.334115 15519 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.189:43587
I20260811 19:12:01.334133 15808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.189:43587 every 8 connection(s)
I20260811 19:12:01.335542 15519 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-1/data/info.pb
I20260811 19:12:01.343019 15810 sys_catalog.cc:263] Verifying existing consensus state
I20260811 19:12:01.347311 15810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0: Bootstrap starting.
I20260811 19:12:01.365902 15810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 19:12:01.366547 15810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0: Bootstrap complete.
I20260811 19:12:01.367705 15810 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: NON_VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: true } }
I20260811 19:12:01.368076 15810 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0c62dbd92db642fa9b56acc74117c8c0, State: Initialized, Role: LEARNER
I20260811 19:12:01.368374 15810 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: NON_VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: true } }
I20260811 19:12:01.369925 15819 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: NON_VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: true } } }
I20260811 19:12:01.369949 15810 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:12:01.370486 15819 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0 [sys.catalog]: This master's current role is: LEARNER
I20260811 19:12:01.385516 15832 catalog_manager.cc:1259] Loaded cluster ID: e24c3dbff427436d95a3345db6017754
I20260811 19:12:01.385710 15832 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0: loading cluster ID for follower catalog manager: success
I20260811 19:12:01.389492 15832 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0: acquiring CA information for follower catalog manager: success
I20260811 19:12:01.392035 15832 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 19:12:01.534767 15683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:12:01.535171 15683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:12:01.535344 15683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:12:01.544467 15683 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:12:01.544646 15683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:12:01.544749 15683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:12:01.544847 15683 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:12:01.544929 15683 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:12:01.557775 15683 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:34789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.3.61.190:33527,127.3.61.189:43587,127.3.61.188:46509
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/krb5kdc/oryx_127.3.61.188.keytab
--principal=oryx/127.3.61.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.188:46509
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.3.61.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:12:01.559409 15683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:12:01.560981 15683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:12:01.576086 15839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:01.576116 15844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:01.576493 15838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:12:01.578042 15683 server_base.cc:1034] running on GCE node
I20260811 19:12:01.579380 15683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:12:01.581465 15683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:12:01.582854 15683 hybrid_clock.cc:648] HybridClock initialized: now 1786475521582740 us; error 102 us; skew 500 ppm
I20260811 19:12:01.583423 15683 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:12:01 dist-test-slave-t7g5 krb5kdc[14712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475521, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.61.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:12:01.591091 15683 init.cc:377] Logged in from keytab as oryx/127.3.61.188@KRBTEST.COM (short username oryx)
I20260811 19:12:01.593238 15683 webserver.cc:460] Webserver started at http://127.3.61.188:36815/ using document root <none> and password file <none>
I20260811 19:12:01.594506 15683 fs_manager.cc:359] Metadata directory not provided
I20260811 19:12:01.594726 15683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:12:01.595142 15683 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:12:01.599732 15683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-2/data/instance:
uuid: "006be47d760e41be8683477553cb2bf3"
format_stamp: "Formatted at 2026-08-11 19:12:01 on dist-test-slave-t7g5"
server_key: "5dc8a099adf0277efe37d39e3d96c913"
server_key_iv: "ea3c7c2ba9912918a87b618806c211e1"
server_key_version: "encryptionkey@0"
I20260811 19:12:01.600769 15683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-2/wal/instance:
uuid: "006be47d760e41be8683477553cb2bf3"
format_stamp: "Formatted at 2026-08-11 19:12:01 on dist-test-slave-t7g5"
server_key: "5dc8a099adf0277efe37d39e3d96c913"
server_key_iv: "ea3c7c2ba9912918a87b618806c211e1"
server_key_version: "encryptionkey@0"
I20260811 19:12:01.609045 15683 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.008s sys 0.002s
I20260811 19:12:01.613905 15858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:01.615794 15683 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260811 19:12:01.616039 15683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-2/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-2/wal
uuid: "006be47d760e41be8683477553cb2bf3"
format_stamp: "Formatted at 2026-08-11 19:12:01 on dist-test-slave-t7g5"
server_key: "5dc8a099adf0277efe37d39e3d96c913"
server_key_iv: "ea3c7c2ba9912918a87b618806c211e1"
server_key_version: "encryptionkey@0"
I20260811 19:12:01.616297 15683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:12:01.637691 15683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:12:01.640478 15865 sys_catalog.cc:455] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d719d69f5e494d2eaecba22f5241c8b4" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: NON_VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: true } } }
I20260811 19:12:01.641106 15865 sys_catalog.cc:458] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4 [sys.catalog]: This master's current role is: LEADER
I20260811 19:12:01.644796 15683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:12:01.645257 15683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:12:01.654351 15872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:01.640562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.189:43587 (local address 127.0.0.1:41828)
0811 19:12:01.646514 (+ 5952us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:01.646537 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:01.646750 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:01.647296 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:01.647300 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:01.647323 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:12:01.647517 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:01.647525 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:01.649883 (+ 2358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:01.649888 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:01.651457 (+ 1569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:01.651476 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:01.653409 (+ 1933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:01.653417 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:01.653687 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:01.653766 (+ 79us) client_negotiation.cc:769] Sending connection context
0811 19:12:01.653889 (+ 123us) client_negotiation.cc:240] Negotiation successful
0811 19:12:01.654023 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5705,"thread_start_us":5445,"threads_started":1}
I20260811 19:12:01.654577 15875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:01.641034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41828 (local address 127.3.61.189:43587)
0811 19:12:01.642399 (+ 1365us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:01.642407 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:01.646668 (+ 4261us) server_negotiation.cc:407] Connection header received
0811 19:12:01.646973 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:01.646979 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:01.647087 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:01.647235 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:12:01.647671 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:01.649599 (+ 1928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:01.651683 (+ 2084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:01.653094 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:01.653270 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:01.653602 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:01.653935 (+ 333us) server_negotiation.cc:299] Negotiation successful
0811 19:12:01.654110 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1205,"thread_start_us":973,"threads_started":1}
I20260811 19:12:01.655957 15819 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d719d69f5e494d2eaecba22f5241c8b4. Latest consensus state: current_term: 1 leader_uuid: "d719d69f5e494d2eaecba22f5241c8b4" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: NON_VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: true } } }
I20260811 19:12:01.656360 15819 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0 [sys.catalog]: This master's current role is: LEARNER
I20260811 19:12:01.657328 15869 raft_consensus.cc:1050] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4: attempting to promote NON_VOTER 0c62dbd92db642fa9b56acc74117c8c0 to VOTER
I20260811 19:12:01.658615 15869 consensus_queue.cc:237] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: false } }
I20260811 19:12:01.661047 15773 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0 [term 1 LEARNER]: Refusing update from remote peer d719d69f5e494d2eaecba22f5241c8b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 19:12:01.661788 15869 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260811 19:12:01.666328 15865 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 0c62dbd92db642fa9b56acc74117c8c0 (127.3.61.189) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: false } } }
I20260811 19:12:01.666802 15773 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 0c62dbd92db642fa9b56acc74117c8c0 (127.3.61.189) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: false } } }
I20260811 19:12:01.671433 15865 sys_catalog.cc:455] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d719d69f5e494d2eaecba22f5241c8b4" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: false } } }
I20260811 19:12:01.671483 15867 sys_catalog.cc:455] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d719d69f5e494d2eaecba22f5241c8b4" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: false } } }
I20260811 19:12:01.671821 15867 sys_catalog.cc:458] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4 [sys.catalog]: This master's current role is: LEADER
I20260811 19:12:01.671814 15865 sys_catalog.cc:458] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4 [sys.catalog]: This master's current role is: LEADER
I20260811 19:12:01.672564 15819 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d719d69f5e494d2eaecba22f5241c8b4" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: false } } }
I20260811 19:12:01.672991 15819 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:12:01.688750 15683 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.188:46509
I20260811 19:12:01.688892 15946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.188:46509 every 8 connection(s)
I20260811 19:12:01.690941 15683 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-2/data/info.pb
I20260811 19:12:01.697003 15948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:12:01.697434 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 15683
I20260811 19:12:01.710456 15948 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 19:12:01 dist-test-slave-t7g5 krb5kdc[14712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475517, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.61.188@KRBTEST.COM
Aug 11 19:12:01 dist-test-slave-t7g5 krb5kdc[14712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475521, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.61.188@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM
I20260811 19:12:01.728523 15952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:01.700607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48508 (local address 127.3.61.188:46509)
0811 19:12:01.702218 (+ 1611us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:01.702236 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:01.702310 (+ 74us) server_negotiation.cc:407] Connection header received
0811 19:12:01.704411 (+ 2101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:01.704441 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:01.704986 (+ 545us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:01.705469 (+ 483us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:01.707466 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:01.709010 (+ 1544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:01.710631 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:01.711537 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:01.711735 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:01.718040 (+ 6305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:01.718052 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:01.718068 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:01.718139 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:01.722406 (+ 4267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:01.723791 (+ 1385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:01.723800 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:01.723810 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:01.724086 (+ 276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:01.724659 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:01.724665 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:01.724672 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:01.725553 (+ 881us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:01.725981 (+ 428us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:01.726653 (+ 672us) server_negotiation.cc:299] Negotiation successful
0811 19:12:01.727081 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1316,"thread_start_us":802,"threads_started":1}
I20260811 19:12:01.736436 15955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:01.714971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58390 (local address 127.3.61.190:33527)
0811 19:12:01.716204 (+ 1233us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:01.716209 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:01.716228 (+ 19us) server_negotiation.cc:407] Connection header received
0811 19:12:01.716294 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:01.716299 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:01.716385 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:01.716511 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:01.718454 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:01.719520 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:01.722179 (+ 2659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:01.723217 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:01.723378 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:01.728317 (+ 4939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:01.728323 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:01.728327 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:01.728367 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:01.732341 (+ 3974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:01.733377 (+ 1036us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:01.733383 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:01.733386 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:01.733725 (+ 339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:01.734311 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:01.734316 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:01.734319 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:01.734766 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:01.734963 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:01.735788 (+ 825us) server_negotiation.cc:299] Negotiation successful
0811 19:12:01.736073 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1087,"thread_start_us":923,"threads_started":1}
I20260811 19:12:01.736548 15954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:01.714198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33527 (local address 127.0.0.1:58390)
0811 19:12:01.715362 (+ 1164us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:01.715409 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:01.715624 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:01.716621 (+ 997us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:01.716630 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:01.717599 (+ 969us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:01.718186 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:01.718204 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:01.719707 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:01.719713 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:01.721802 (+ 2089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:01.721848 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:01.723455 (+ 1607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:01.723471 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:01.723623 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:01.724406 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:01.724426 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:01.728051 (+ 3625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:12:01.732524 (+ 4473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:01.732538 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:01.732543 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:01.733112 (+ 569us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:01.733895 (+ 783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:01.733899 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:01.733902 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:01.734091 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:01.735057 (+ 966us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:12:01.735066 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:12:01.735320 (+ 254us) client_negotiation.cc:769] Sending connection context
0811 19:12:01.735601 (+ 281us) client_negotiation.cc:240] Negotiation successful
0811 19:12:01.736082 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":854,"thread_start_us":642,"threads_started":1}
I20260811 19:12:01.738467 15948 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43587 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 19:12:01 dist-test-slave-t7g5 krb5kdc[14712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475521, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.61.188@KRBTEST.COM for oryx/127.3.61.189@KRBTEST.COM
I20260811 19:12:01.755909 15954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:01.739400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.189:43587 (local address 127.0.0.1:41832)
0811 19:12:01.739701 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:01.739722 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:01.739843 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:01.740161 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:01.740166 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:01.740635 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:01.740881 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:01.740892 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:01.742173 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:01.742180 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:01.743391 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:01.743407 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:01.744512 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:01.744517 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:01.744951 (+ 434us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:01.745897 (+ 946us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:01.745930 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:01.748707 (+ 2777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:12:01.753041 (+ 4334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:01.753046 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:01.753049 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:01.753667 (+ 618us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:01.754264 (+ 597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:01.754268 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:01.754271 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:01.754436 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:01.755025 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:12:01.755029 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:12:01.755181 (+ 152us) client_negotiation.cc:769] Sending connection context
0811 19:12:01.755300 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 19:12:01.755610 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102}
I20260811 19:12:01.756086 15875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:01.739519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41832 (local address 127.3.61.189:43587)
0811 19:12:01.739743 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:01.739748 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:01.739770 (+ 22us) server_negotiation.cc:407] Connection header received
0811 19:12:01.739970 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:01.739974 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:01.740036 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:01.740121 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:01.741083 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:01.741955 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:01.743693 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:01.744291 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:01.744419 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:01.749031 (+ 4612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:01.749038 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:01.749042 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:01.749091 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:01.752803 (+ 3712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:01.753851 (+ 1048us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:01.753857 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:01.753859 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:01.754109 (+ 250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:01.754581 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:01.754586 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:01.754588 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:01.754906 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:01.755033 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:01.755484 (+ 451us) server_negotiation.cc:299] Negotiation successful
0811 19:12:01.755733 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 19:12:01.757540 15948 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.61.188" port: 46509 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 19:12:01 dist-test-slave-t7g5 krb5kdc[14712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475521, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.61.188@KRBTEST.COM for oryx/127.3.61.188@KRBTEST.COM
I20260811 19:12:01.776124 15954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:01.758394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:46509 (local address 127.0.0.1:48510)
0811 19:12:01.758695 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:01.758719 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:01.758846 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:01.759319 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:01.759324 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:01.759924 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:01.760470 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:01.760486 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:01.761735 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:01.761739 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:01.762806 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:01.762818 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:01.763830 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:01.763836 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:01.764100 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:01.764901 (+ 801us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:01.765005 (+ 104us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:01.767647 (+ 2642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:12:01.771960 (+ 4313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:01.771967 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:01.771971 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:01.772685 (+ 714us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:01.773235 (+ 550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:01.773239 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:01.773242 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:01.773491 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:01.774473 (+ 982us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:12:01.774477 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:12:01.774699 (+ 222us) client_negotiation.cc:769] Sending connection context
0811 19:12:01.774882 (+ 183us) client_negotiation.cc:240] Negotiation successful
0811 19:12:01.775412 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86}
I20260811 19:12:01.776351 15952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:01.758746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48510 (local address 127.3.61.188:46509)
0811 19:12:01.758928 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:01.758932 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:01.758948 (+ 16us) server_negotiation.cc:407] Connection header received
0811 19:12:01.758989 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:01.758992 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:01.759047 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:01.759130 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:01.760639 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:01.761410 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:01.763008 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:01.763535 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:01.763678 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:01.767905 (+ 4227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:01.767914 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:01.767919 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:01.767976 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:01.771708 (+ 3732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:01.772850 (+ 1142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:01.772857 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:01.772859 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:01.773088 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:01.773802 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:01.773816 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:01.773819 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:01.774304 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:01.774466 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:01.775133 (+ 667us) server_negotiation.cc:299] Negotiation successful
0811 19:12:01.775779 (+ 646us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":178,"server-negotiator.queue_time_us":78}
I20260811 19:12:01.782688 15948 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3: Bootstrap starting.
I20260811 19:12:01.787009 15948 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3: Neither blocks nor log segments found. Creating new log.
I20260811 19:12:01.788769 15948 log.cc:824] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3: Log is configured to *not* fsync() on all Append() calls
I20260811 19:12:01.792989 15948 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3: No bootstrap required, opened a new log
I20260811 19:12:01.802501 15948 raft_consensus.cc:348] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43587 } } peers { permanent_uuid: "006be47d760e41be8683477553cb2bf3" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 46509 } }
I20260811 19:12:01.802953 15948 raft_consensus.cc:374] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:12:01.803097 15948 raft_consensus.cc:729] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 006be47d760e41be8683477553cb2bf3, State: Initialized, Role: FOLLOWER
I20260811 19:12:01.803778 15948 consensus_queue.cc:260] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43587 } } peers { permanent_uuid: "006be47d760e41be8683477553cb2bf3" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 46509 } }
I20260811 19:12:01.804903 15958 sys_catalog.cc:455] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43587 } } peers { permanent_uuid: "006be47d760e41be8683477553cb2bf3" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 46509 } } }
I20260811 19:12:01.805581 15958 sys_catalog.cc:458] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:12:01.806979 15948 sys_catalog.cc:564] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 19:12:01.821615 15979 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 19:12:01.821864 15979 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 19:12:01.828193 15683 master_runner.cc:428] Detected that this master 006be47d760e41be8683477553cb2bf3 is joining an existing cluster
I20260811 19:12:01.828460 15683 master_runner.cc:432] Initiating AddMaster RPC to add 127.3.61.188:46509
I20260811 19:12:01.836339 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 14880
I20260811 19:12:01.860513 15994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:01.835901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.189:43587 (local address 127.0.0.1:41840)
0811 19:12:01.838358 (+ 2457us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:01.838379 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:01.838543 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:01.839144 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:01.839150 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:01.840588 (+ 1438us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:01.841603 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:01.841619 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:01.843533 (+ 1914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:01.843541 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:01.846178 (+ 2637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:01.846202 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:01.847868 (+ 1666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:01.847875 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:01.848111 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:01.849286 (+ 1175us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:01.849315 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:01.851070 (+ 1755us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:12:01.856526 (+ 5456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:01.856533 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:01.856538 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:01.857199 (+ 661us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:01.857783 (+ 584us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:01.857788 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:01.857792 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:01.858118 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:01.858963 (+ 845us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:12:01.858969 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:12:01.859259 (+ 290us) client_negotiation.cc:769] Sending connection context
0811 19:12:01.859426 (+ 167us) client_negotiation.cc:240] Negotiation successful
0811 19:12:01.859861 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2262,"thread_start_us":2016,"threads_started":1}
I20260811 19:12:01.861948 15875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:01.835575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41840 (local address 127.3.61.189:43587)
0811 19:12:01.835783 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:01.835789 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:01.838493 (+ 2704us) server_negotiation.cc:407] Connection header received
0811 19:12:01.838677 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:01.838683 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:01.838781 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:01.838900 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:01.841875 (+ 2975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:01.843194 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:01.846431 (+ 3237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:01.847313 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:01.847424 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:01.851347 (+ 3923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:01.851355 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:01.851361 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:01.851410 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:01.856265 (+ 4855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:01.857374 (+ 1109us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:01.857381 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:01.857385 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:01.857605 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:01.858263 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:01.858269 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:01.858273 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:01.858733 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:01.858888 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:01.861099 (+ 2211us) server_negotiation.cc:299] Negotiation successful
0811 19:12:01.861419 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
Aug 11 19:12:01 dist-test-slave-t7g5 krb5kdc[14712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475517, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.61.129@KRBTEST.COM
I20260811 19:12:01.878777 15996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:01.837705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55514 (local address 127.3.61.129:38449)
0811 19:12:01.850077 (+ 12372us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:01.850087 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:01.850123 (+ 36us) server_negotiation.cc:407] Connection header received
0811 19:12:01.850205 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:01.850216 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:01.850557 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:01.853337 (+ 2780us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:01.853684 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:01.855881 (+ 2197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:01.858496 (+ 2615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:01.859599 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:01.866957 (+ 7358us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:01.867364 (+ 407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:01.867379 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:01.867402 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:01.867480 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:01.872666 (+ 5186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:01.874332 (+ 1666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:01.874343 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:01.874346 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:01.874615 (+ 269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:01.875260 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:01.875266 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:01.875270 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:01.875812 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:01.877552 (+ 1740us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:01.877796 (+ 244us) server_negotiation.cc:299] Negotiation successful
0811 19:12:01.878118 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12057,"thread_start_us":12980,"threads_started":1}
I20260811 19:12:01.880775 15086 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:12:01.881604 15955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:01.835145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58404 (local address 127.3.61.190:33527)
0811 19:12:01.835376 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:01.835383 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:01.839426 (+ 4043us) server_negotiation.cc:407] Connection header received
0811 19:12:01.839718 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:01.839724 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:01.839857 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:01.839939 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:01.864211 (+ 24272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:01.865573 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:01.868758 (+ 3185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:01.869509 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:01.869591 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:01.873006 (+ 3415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:01.873013 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:01.873018 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:01.873060 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:01.877977 (+ 4917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:01.878887 (+ 910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:01.878892 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:01.878896 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:01.879084 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:01.879605 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:01.879611 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:01.879614 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:01.880033 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:01.880185 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:01.880941 (+ 756us) server_negotiation.cc:299] Negotiation successful
0811 19:12:01.881204 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 19:12:01.883203 15995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:01.836502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33527 (local address 127.0.0.1:58404)
0811 19:12:01.839321 (+ 2819us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:01.839340 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:01.839558 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:01.840187 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:01.840192 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:01.841819 (+ 1627us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:01.863106 (+ 21287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:01.863128 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:01.865907 (+ 2779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:01.865912 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:01.868565 (+ 2653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:01.868579 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:01.870066 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:01.870075 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:01.870312 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:01.871086 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:01.871114 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:01.872667 (+ 1553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:12:01.878154 (+ 5487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:01.878163 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:01.878167 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:01.878732 (+ 565us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:01.879239 (+ 507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:01.879244 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:01.879247 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:01.879467 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:01.880298 (+ 831us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:12:01.880303 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:12:01.880592 (+ 289us) client_negotiation.cc:769] Sending connection context
0811 19:12:01.882373 (+ 1781us) client_negotiation.cc:240] Negotiation successful
0811 19:12:01.882718 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2659,"mutex_wait_us":21,"thread_start_us":2435,"threads_started":1}
I20260811 19:12:01.887638 15952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:01.839505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48516 (local address 127.3.61.188:46509)
0811 19:12:01.839719 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:01.839725 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:01.863954 (+ 24229us) server_negotiation.cc:407] Connection header received
0811 19:12:01.864232 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:01.864236 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:01.864305 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:01.864379 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:01.866784 (+ 2405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:01.868116 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:01.872255 (+ 4139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:01.873100 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:01.873198 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:01.876405 (+ 3207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:01.876413 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:01.876417 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:01.876454 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:01.881543 (+ 5089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:01.884589 (+ 3046us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:01.884594 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:01.884598 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:01.884781 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:01.885200 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:01.885205 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:01.885208 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:01.885769 (+ 561us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:01.885960 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:01.886593 (+ 633us) server_negotiation.cc:299] Negotiation successful
0811 19:12:01.886841 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 19:12:01.889740 15997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:01.839656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:46509 (local address 127.0.0.1:48516)
0811 19:12:01.863830 (+ 24174us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:01.863847 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:01.864049 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:01.864753 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:01.864758 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:01.865692 (+ 934us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:01.866405 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:01.866422 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:01.869419 (+ 2997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:01.869424 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:01.872062 (+ 2638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:01.872081 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:01.873778 (+ 1697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:01.873787 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:01.874019 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:01.874696 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:01.874720 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:01.876190 (+ 1470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:12:01.883869 (+ 7679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:01.883875 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:01.883878 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:01.884451 (+ 573us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:01.884906 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:01.884911 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:01.884914 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:01.885092 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:01.886029 (+ 937us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:12:01.886034 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:12:01.886287 (+ 253us) client_negotiation.cc:769] Sending connection context
0811 19:12:01.888136 (+ 1849us) client_negotiation.cc:240] Negotiation successful
0811 19:12:01.888886 (+ 750us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24047,"mutex_wait_us":3,"thread_start_us":23843,"threads_started":1}
W20260811 19:12:01.898754 15878 tablet.cc:2369] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 19:12:01 dist-test-slave-t7g5 krb5kdc[14712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475518, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.61.190@KRBTEST.COM for oryx/127.3.61.188@KRBTEST.COM
I20260811 19:12:01.910902 15872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:01.890486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:46509 (local address 127.0.0.1:48532)
0811 19:12:01.890887 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:01.890907 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:01.891019 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:01.891468 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:01.891474 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:01.892091 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:01.892336 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:01.892349 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:01.894479 (+ 2130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:01.894486 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:01.896136 (+ 1650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:01.896155 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:01.897731 (+ 1576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:01.897738 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:01.897989 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:01.898833 (+ 844us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:01.898856 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:01.902290 (+ 3434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:12:01.907643 (+ 5353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:01.907650 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:01.907654 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:01.908353 (+ 699us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:01.908820 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:01.908825 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:01.908829 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:01.909027 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:01.909739 (+ 712us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:12:01.909745 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:12:01.909989 (+ 244us) client_negotiation.cc:769] Sending connection context
0811 19:12:01.910162 (+ 173us) client_negotiation.cc:240] Negotiation successful
0811 19:12:01.910486 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256}
I20260811 19:12:01.910923 15952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:01.890863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48532 (local address 127.3.61.188:46509)
0811 19:12:01.891029 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:01.891034 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:01.891053 (+ 19us) server_negotiation.cc:407] Connection header received
0811 19:12:01.891174 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:01.891179 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:01.891266 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:01.891374 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:01.892696 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:01.894228 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:01.896369 (+ 2141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:01.897506 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:01.897687 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:01.902576 (+ 4889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:01.902584 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:01.902590 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:01.902633 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:01.907449 (+ 4816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:01.908508 (+ 1059us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:01.908519 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:01.908523 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:01.908697 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:01.909142 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:01.909147 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:01.909151 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:01.909594 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:01.909784 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:01.910260 (+ 476us) server_negotiation.cc:299] Negotiation successful
0811 19:12:01.910508 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 19:12:01.913101 14792 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.3.61.188:46509: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "006be47d760e41be8683477553cb2bf3"
member_type: NON_VOTER
last_known_addr {
host: "127.3.61.188"
port: 46509
}
attrs {
promote: true
}
}
dest_uuid: "d719d69f5e494d2eaecba22f5241c8b4"
cas_config_opid_index: 6
I20260811 19:12:01.914059 14792 consensus_queue.cc:237] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: false } } peers { permanent_uuid: "006be47d760e41be8683477553cb2bf3" member_type: NON_VOTER last_known_addr { host: "127.3.61.188" port: 46509 } attrs { promote: true } }
I20260811 19:12:01.916546 15773 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0 [term 1 FOLLOWER]: Refusing update from remote peer d719d69f5e494d2eaecba22f5241c8b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260811 19:12:01.916997 15909 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:12:01.918298 15867 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260811 19:12:01.921635 15909 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [term 1 FOLLOWER]: Refusing update from remote peer d719d69f5e494d2eaecba22f5241c8b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260811 19:12:01.922639 15869 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "006be47d760e41be8683477553cb2bf3" member_type: NON_VOTER last_known_addr { host: "127.3.61.188" port: 46509 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:12:01.923065 15865 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 006be47d760e41be8683477553cb2bf3 (127.3.61.188) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: false } } peers { permanent_uuid: "006be47d760e41be8683477553cb2bf3" member_type: NON_VOTER last_known_addr { host: "127.3.61.188" port: 46509 } attrs { promote: true } } }
I20260811 19:12:01.923870 15773 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 006be47d760e41be8683477553cb2bf3 (127.3.61.188) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: false } } peers { permanent_uuid: "006be47d760e41be8683477553cb2bf3" member_type: NON_VOTER last_known_addr { host: "127.3.61.188" port: 46509 } attrs { promote: true } } }
I20260811 19:12:01.928292 15865 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.3.61.188:46509
I20260811 19:12:01.928912 15819 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d719d69f5e494d2eaecba22f5241c8b4" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: false } } peers { permanent_uuid: "006be47d760e41be8683477553cb2bf3" member_type: NON_VOTER last_known_addr { host: "127.3.61.188" port: 46509 } attrs { promote: true } } }
I20260811 19:12:01.929282 15819 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:12:01.929555 15683 master.cc:537] Master@127.3.61.188:46509 shutting down...
I20260811 19:12:01.930063 15869 sys_catalog.cc:455] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d719d69f5e494d2eaecba22f5241c8b4" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: false } } peers { permanent_uuid: "006be47d760e41be8683477553cb2bf3" member_type: NON_VOTER last_known_addr { host: "127.3.61.188" port: 46509 } attrs { promote: true } } }
I20260811 19:12:01.930486 15869 sys_catalog.cc:458] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4 [sys.catalog]: This master's current role is: LEADER
I20260811 19:12:01.931043 15865 sys_catalog.cc:455] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d719d69f5e494d2eaecba22f5241c8b4" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: false } } peers { permanent_uuid: "006be47d760e41be8683477553cb2bf3" member_type: NON_VOTER last_known_addr { host: "127.3.61.188" port: 46509 } attrs { promote: true } } }
I20260811 19:12:01.931624 15865 sys_catalog.cc:458] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4 [sys.catalog]: This master's current role is: LEADER
I20260811 19:12:01.943378 15909 raft_consensus.cc:858] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43587 } } peers { permanent_uuid: "006be47d760e41be8683477553cb2bf3" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 46509 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: NON_VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: false } }
W20260811 19:12:01.943598 15909 raft_consensus.cc:1565] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 1 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43587 } } peers { permanent_uuid: "006be47d760e41be8683477553cb2bf3" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 46509 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: NON_VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: false } }
I20260811 19:12:01.953526 15909 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 0c62dbd92db642fa9b56acc74117c8c0 (127.3.61.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: NON_VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: true } } }
I20260811 19:12:01.975515 15958 sys_catalog.cc:455] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d719d69f5e494d2eaecba22f5241c8b4. Latest consensus state: current_term: 1 leader_uuid: "d719d69f5e494d2eaecba22f5241c8b4" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: NON_VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: true } } }
I20260811 19:12:01.975805 15958 sys_catalog.cc:458] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260811 19:12:01.974351 16013 sys_catalog.cc:455] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d719d69f5e494d2eaecba22f5241c8b4" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: NON_VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: true } } }
I20260811 19:12:01.976310 16013 sys_catalog.cc:458] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260811 19:12:01.976966 15869 sys_catalog.cc:455] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d719d69f5e494d2eaecba22f5241c8b4" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: false } } peers { permanent_uuid: "006be47d760e41be8683477553cb2bf3" member_type: NON_VOTER last_known_addr { host: "127.3.61.188" port: 46509 } attrs { promote: true } } }
I20260811 19:12:01.977332 15869 sys_catalog.cc:458] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4 [sys.catalog]: This master's current role is: LEADER
I20260811 19:12:01.977015 16007 sys_catalog.cc:455] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d719d69f5e494d2eaecba22f5241c8b4" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: NON_VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: true } } }
I20260811 19:12:01.977526 16007 sys_catalog.cc:458] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [sys.catalog]: This master's current role is: NON_PARTICIPANT
W20260811 19:12:01.979194 14772 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260811 19:12:01.980674 15683 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260811 19:12:01.981222 15683 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260811 19:12:01.981397 15683 tablet_replica.cc:330] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3: stopping tablet replica
I20260811 19:12:01.991088 15872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:01.990548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:46509 (local address 127.0.0.1:48546)
0811 19:12:01.990715 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:01.990869 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.188:46509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 19:12:01.991482 14772 consensus_peers.cc:487] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4 -> Peer 006be47d760e41be8683477553cb2bf3 (127.3.61.188:46509): Couldn't send request to peer 006be47d760e41be8683477553cb2bf3. Status: Network error: Client connection negotiation failed: client connection to 127.3.61.188:46509: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 19:12:02.002489 15683 master.cc:552] Master@127.3.61.188:46509 shutdown complete.
I20260811 19:12:02.002609 15683 master_runner.cc:305] Clearing existing system tablet
I20260811 19:12:02.003047 15683 fs_manager.cc:359] Metadata directory not provided
I20260811 19:12:02.003170 15683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:12:02.006712 15683 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.004s
I20260811 19:12:02.008949 16016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:02.009618 15683 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:12:02.010263 15683 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 19:12:02.010396 15683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-2/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-2/wal
uuid: "006be47d760e41be8683477553cb2bf3"
format_stamp: "Formatted at 2026-08-11 19:12:01 on dist-test-slave-t7g5"
server_key: "5dc8a099adf0277efe37d39e3d96c913"
server_key_iv: "ea3c7c2ba9912918a87b618806c211e1"
server_key_version: "encryptionkey@0"
I20260811 19:12:02.011479 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 14880
I20260811 19:12:02.016986 15683 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:12:02.025164 15683 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 19:12:02.025384 15683 log.cc:1196] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260811 19:12:02.025969 15683 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3: Deleting consensus metadata
I20260811 19:12:02.026268 15683 master_runner.cc:315] Copying system tablet from 127.3.61.190:33527
I20260811 19:12:02.030525 15683 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3: tablet copy: Beginning tablet copy session from remote peer at address 127.3.61.190:33527
I20260811 19:12:02.046521 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.129:38449
--local_ip_for_outbound_sockets=127.3.61.129
--tserver_master_addrs=127.3.61.190:33527
--webserver_port=37447
--webserver_interface=127.3.61.129
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/krb5kdc/oryx_127.3.61.129.keytab
--principal=oryx/127.3.61.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:34789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.3.61.190:33527,127.3.61.189:43587
--tserver_master_addrs=127.3.61.190:33527,127.3.61.189:43587,127.3.61.188:46509 with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:12:02.050283 15955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:02.031616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58414 (local address 127.3.61.190:33527)
0811 19:12:02.031858 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:02.031864 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:02.034243 (+ 2379us) server_negotiation.cc:407] Connection header received
0811 19:12:02.034486 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:02.034493 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:02.034590 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:02.034699 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:02.036048 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:02.037336 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:02.039376 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:02.040033 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:02.040169 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:02.042793 (+ 2624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:02.042800 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:02.042805 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:02.042849 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:02.046324 (+ 3475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:02.047194 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:02.047201 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:02.047204 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:02.047350 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:02.047771 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:02.047775 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:02.047777 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:02.048417 (+ 640us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:02.048735 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:02.049468 (+ 733us) server_negotiation.cc:299] Negotiation successful
0811 19:12:02.049779 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
I20260811 19:12:02.050688 16024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:02.031564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33527 (local address 127.0.0.1:58414)
0811 19:12:02.034072 (+ 2508us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:02.034101 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:02.034291 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:02.034839 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:02.034845 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:02.035495 (+ 650us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:02.035804 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:02.035842 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:02.037512 (+ 1670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:02.037519 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:02.039189 (+ 1670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:02.039208 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:02.040212 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:02.040219 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:02.040426 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:02.041169 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:02.041194 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:02.042526 (+ 1332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:12:02.046510 (+ 3984us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:02.046514 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:02.046518 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:02.047052 (+ 534us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:02.047478 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:02.047483 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:02.047487 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:02.047647 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:02.048697 (+ 1050us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:12:02.048711 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:12:02.048987 (+ 276us) client_negotiation.cc:769] Sending connection context
0811 19:12:02.049274 (+ 287us) client_negotiation.cc:240] Negotiation successful
0811 19:12:02.049957 (+ 683us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2335,"thread_start_us":2215,"threads_started":1}
I20260811 19:12:02.051614 14824 tablet_copy_service.cc:140] P d719d69f5e494d2eaecba22f5241c8b4: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 006be47d760e41be8683477553cb2bf3 ({username='oryx', principal='oryx/127.3.61.188@KRBTEST.COM'} at 127.0.0.1:58414)
I20260811 19:12:02.051928 14824 tablet_copy_service.cc:161] P d719d69f5e494d2eaecba22f5241c8b4: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 006be47d760e41be8683477553cb2bf3 at {username='oryx', principal='oryx/127.3.61.188@KRBTEST.COM'} at 127.0.0.1:58414: session id = 006be47d760e41be8683477553cb2bf3-00000000000000000000000000000000
I20260811 19:12:02.054850 14824 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 19:12:02.057260 15683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 19:12:02.063802 15683 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3: tablet copy: Starting download of 0 data blocks...
I20260811 19:12:02.064260 15683 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3: tablet copy: Starting download of 1 WAL segments...
I20260811 19:12:02.068419 15683 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 19:12:02.074297 14824 tablet_copy_service.cc:342] P d719d69f5e494d2eaecba22f5241c8b4: Request end of tablet copy session 006be47d760e41be8683477553cb2bf3-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.3.61.188@KRBTEST.COM'} at 127.0.0.1:58414
I20260811 19:12:02.074486 14824 tablet_copy_service.cc:433] P d719d69f5e494d2eaecba22f5241c8b4: ending tablet copy session 006be47d760e41be8683477553cb2bf3-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 006be47d760e41be8683477553cb2bf3
I20260811 19:12:02.088106 15683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:12:02.091058 16026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:02.092190 16027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:02.099745 16030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:12:02.100401 15683 server_base.cc:1034] running on GCE node
I20260811 19:12:02.100843 15683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:12:02.101641 15683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:12:02.102944 15683 hybrid_clock.cc:648] HybridClock initialized: now 1786475522102895 us; error 43 us; skew 500 ppm
I20260811 19:12:02.103299 15683 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:12:02 dist-test-slave-t7g5 krb5kdc[14712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475522, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.61.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:12:02.105680 15683 init.cc:377] Logged in from keytab as oryx/127.3.61.188@KRBTEST.COM (short username oryx)
I20260811 19:12:02.107187 15683 webserver.cc:460] Webserver started at http://127.3.61.188:37587/ using document root <none> and password file <none>
I20260811 19:12:02.107590 15683 fs_manager.cc:359] Metadata directory not provided
I20260811 19:12:02.107764 15683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:12:02.111073 15683 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.002s
I20260811 19:12:02.113313 16038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:02.113904 15683 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260811 19:12:02.114084 15683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-2/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-2/wal
uuid: "006be47d760e41be8683477553cb2bf3"
format_stamp: "Formatted at 2026-08-11 19:12:01 on dist-test-slave-t7g5"
server_key: "5dc8a099adf0277efe37d39e3d96c913"
server_key_iv: "ea3c7c2ba9912918a87b618806c211e1"
server_key_version: "encryptionkey@0"
I20260811 19:12:02.114349 15683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:12:02.123910 15683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:12:02.128965 15683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:12:02.160897 15683 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.188:46509
I20260811 19:12:02.160960 16121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.188:46509 every 8 connection(s)
I20260811 19:12:02.162307 15683 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/master-2/data/info.pb
I20260811 19:12:02.168455 16123 sys_catalog.cc:263] Verifying existing consensus state
I20260811 19:12:02.172700 16123 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3: Bootstrap starting.
I20260811 19:12:02.190824 16123 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 19:12:02.191385 16123 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3: Bootstrap complete.
I20260811 19:12:02.192631 16123 raft_consensus.cc:348] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: false } } peers { permanent_uuid: "006be47d760e41be8683477553cb2bf3" member_type: NON_VOTER last_known_addr { host: "127.3.61.188" port: 46509 } attrs { promote: true } }
I20260811 19:12:02.192945 16123 raft_consensus.cc:729] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 006be47d760e41be8683477553cb2bf3, State: Initialized, Role: LEARNER
I20260811 19:12:02.193250 16123 consensus_queue.cc:260] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: false } } peers { permanent_uuid: "006be47d760e41be8683477553cb2bf3" member_type: NON_VOTER last_known_addr { host: "127.3.61.188" port: 46509 } attrs { promote: true } }
I20260811 19:12:02.198735 16131 sys_catalog.cc:455] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: false } } peers { permanent_uuid: "006be47d760e41be8683477553cb2bf3" member_type: NON_VOTER last_known_addr { host: "127.3.61.188" port: 46509 } attrs { promote: true } } }
I20260811 19:12:02.199420 16131 sys_catalog.cc:458] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [sys.catalog]: This master's current role is: LEARNER
I20260811 19:12:02.199193 16123 sys_catalog.cc:564] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:12:02.281648 16144 catalog_manager.cc:1259] Loaded cluster ID: e24c3dbff427436d95a3345db6017754
I20260811 19:12:02.281818 16144 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3: loading cluster ID for follower catalog manager: success
I20260811 19:12:02.285717 16144 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3: acquiring CA information for follower catalog manager: success
I20260811 19:12:02.288980 16144 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 19:12:02.391592 15869 sys_catalog.cc:455] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d719d69f5e494d2eaecba22f5241c8b4" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: false } } peers { permanent_uuid: "006be47d760e41be8683477553cb2bf3" member_type: NON_VOTER last_known_addr { host: "127.3.61.188" port: 46509 } attrs { promote: true } } }
I20260811 19:12:02.391976 15869 sys_catalog.cc:458] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4 [sys.catalog]: This master's current role is: LEADER
I20260811 19:12:02.401376 15872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:02.391891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:46509 (local address 127.0.0.1:48558)
0811 19:12:02.392163 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:02.392183 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:02.392315 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:02.393783 (+ 1468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:02.393796 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:02.393829 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:12:02.394045 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:02.394056 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:02.396465 (+ 2409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:02.396472 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:02.398413 (+ 1941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:02.398440 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:02.400604 (+ 2164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:02.400612 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:02.400840 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:02.400902 (+ 62us) client_negotiation.cc:769] Sending connection context
0811 19:12:02.401005 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 19:12:02.401138 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132}
I20260811 19:12:02.401693 16146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:02.392669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48558 (local address 127.3.61.188:46509)
0811 19:12:02.393343 (+ 674us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:02.393350 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:02.393379 (+ 29us) server_negotiation.cc:407] Connection header received
0811 19:12:02.393463 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:02.393468 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:02.393566 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:02.393752 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:12:02.394208 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:02.396220 (+ 2012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:02.398710 (+ 2490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:02.400298 (+ 1588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:02.400504 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:02.400999 (+ 495us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:02.401085 (+ 86us) server_negotiation.cc:299] Negotiation successful
0811 19:12:02.401280 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":520,"thread_start_us":332,"threads_started":1}
I20260811 19:12:02.402674 16082 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.5->[1.6-1.7] Dedup: 1.7->[]
I20260811 19:12:02.403151 16131 sys_catalog.cc:455] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d719d69f5e494d2eaecba22f5241c8b4. Latest consensus state: current_term: 1 leader_uuid: "d719d69f5e494d2eaecba22f5241c8b4" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: false } } peers { permanent_uuid: "006be47d760e41be8683477553cb2bf3" member_type: NON_VOTER last_known_addr { host: "127.3.61.188" port: 46509 } attrs { promote: true } } }
I20260811 19:12:02.403468 16131 sys_catalog.cc:458] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [sys.catalog]: This master's current role is: LEARNER
I20260811 19:12:02.403870 15865 raft_consensus.cc:1050] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4: attempting to promote NON_VOTER 006be47d760e41be8683477553cb2bf3 to VOTER
I20260811 19:12:02.404501 15865 consensus_queue.cc:237] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: false } } peers { permanent_uuid: "006be47d760e41be8683477553cb2bf3" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 46509 } attrs { promote: false } }
I20260811 19:12:02.406421 16082 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [term 1 LEARNER]: Refusing update from remote peer d719d69f5e494d2eaecba22f5241c8b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260811 19:12:02.406502 15773 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0 [term 1 FOLLOWER]: Refusing update from remote peer d719d69f5e494d2eaecba22f5241c8b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260811 19:12:02.407032 15867 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260811 19:12:02.407524 15867 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "006be47d760e41be8683477553cb2bf3" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 46509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260811 19:12:02.410415 15867 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 006be47d760e41be8683477553cb2bf3 (127.3.61.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: false } } peers { permanent_uuid: "006be47d760e41be8683477553cb2bf3" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 46509 } attrs { promote: false } } }
I20260811 19:12:02.411448 15773 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 006be47d760e41be8683477553cb2bf3 (127.3.61.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: false } } peers { permanent_uuid: "006be47d760e41be8683477553cb2bf3" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 46509 } attrs { promote: false } } }
I20260811 19:12:02.415994 15865 sys_catalog.cc:455] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d719d69f5e494d2eaecba22f5241c8b4" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: false } } peers { permanent_uuid: "006be47d760e41be8683477553cb2bf3" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 46509 } attrs { promote: false } } }
I20260811 19:12:02.416057 15867 sys_catalog.cc:455] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d719d69f5e494d2eaecba22f5241c8b4" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: false } } peers { permanent_uuid: "006be47d760e41be8683477553cb2bf3" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 46509 } attrs { promote: false } } }
I20260811 19:12:02.416515 15865 sys_catalog.cc:458] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4 [sys.catalog]: This master's current role is: LEADER
I20260811 19:12:02.416695 15867 sys_catalog.cc:458] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4 [sys.catalog]: This master's current role is: LEADER
I20260811 19:12:02.417228 15819 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d719d69f5e494d2eaecba22f5241c8b4" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: false } } peers { permanent_uuid: "006be47d760e41be8683477553cb2bf3" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 46509 } attrs { promote: false } } }
I20260811 19:12:02.417570 15819 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:12:02.423144 15867 sys_catalog.cc:455] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d719d69f5e494d2eaecba22f5241c8b4" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: false } } peers { permanent_uuid: "006be47d760e41be8683477553cb2bf3" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 46509 } attrs { promote: false } } }
I20260811 19:12:02.423539 15867 sys_catalog.cc:458] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4 [sys.catalog]: This master's current role is: LEADER
I20260811 19:12:02.423959 16082 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 006be47d760e41be8683477553cb2bf3 (127.3.61.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: false } } peers { permanent_uuid: "006be47d760e41be8683477553cb2bf3" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 46509 } attrs { promote: false } } }
I20260811 19:12:02.428079 16131 sys_catalog.cc:455] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d719d69f5e494d2eaecba22f5241c8b4" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: false } } peers { permanent_uuid: "006be47d760e41be8683477553cb2bf3" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 46509 } attrs { promote: false } } }
I20260811 19:12:02.428505 16131 sys_catalog.cc:458] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [sys.catalog]: This master's current role is: FOLLOWER
W20260811 19:12:02.557621 16025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:12:02.558065 16025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:12:02.558264 16025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:12:02.568257 16025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:12:02.568435 16025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:12:02.568671 16025 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.129
I20260811 19:12:02.582115 16025 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:34789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/krb5kdc/oryx_127.3.61.129.keytab
--principal=oryx/127.3.61.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.129:38449
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.61.129
--webserver_port=37447
--tserver_master_addrs=127.3.61.190:33527,127.3.61.189:43587,127.3.61.188:46509
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.129
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:12:02.584148 16025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:12:02.586230 16025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:12:02.603639 16156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:02.603781 16155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:02.603689 16159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:12:02.605208 16025 server_base.cc:1034] running on GCE node
I20260811 19:12:02.606057 16025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:12:02.607455 16025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:12:02.608733 16025 hybrid_clock.cc:648] HybridClock initialized: now 1786475522608648 us; error 59 us; skew 500 ppm
I20260811 19:12:02.609136 16025 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:12:02 dist-test-slave-t7g5 krb5kdc[14712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475522, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.61.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:12:02.614873 16025 init.cc:377] Logged in from keytab as oryx/127.3.61.129@KRBTEST.COM (short username oryx)
I20260811 19:12:02.616829 16025 webserver.cc:460] Webserver started at http://127.3.61.129:37447/ using document root <none> and password file <none>
I20260811 19:12:02.617751 16025 fs_manager.cc:359] Metadata directory not provided
I20260811 19:12:02.617969 16025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:12:02.624881 16025 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 19:12:02.629122 16169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:02.630865 16025 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 19:12:02.631079 16025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-0/wal
uuid: "3e21de71b6e9439f80d484412bed0c8a"
format_stamp: "Formatted at 2026-08-11 19:11:58 on dist-test-slave-t7g5"
server_key: "c0229b07bdc403d36a8fdf8326d40940"
server_key_iv: "ed6c7dbb86fd0756613af4007f7ea4b9"
server_key_version: "encryptionkey@0"
I20260811 19:12:02.632388 16025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:12:02.650879 16025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:12:02.656868 16025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:12:02.657198 16025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:12:02.658911 16025 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:12:02.660665 16025 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:12:02.660777 16025 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:02.660882 16025 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:12:02.660970 16025 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:02.720212 16025 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.129:38449
I20260811 19:12:02.720322 16357 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.129:38449 every 8 connection(s)
I20260811 19:12:02.721966 16025 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
I20260811 19:12:02.724730 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 16025
I20260811 19:12:02.725174 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 15100
Aug 11 19:12:02 dist-test-slave-t7g5 krb5kdc[14712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475517, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.61.130@KRBTEST.COM
Aug 11 19:12:02 dist-test-slave-t7g5 krb5kdc[14712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475522, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.61.129@KRBTEST.COM for oryx/127.3.61.188@KRBTEST.COM
Aug 11 19:12:02 dist-test-slave-t7g5 krb5kdc[14712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475522, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.61.129@KRBTEST.COM for oryx/127.3.61.189@KRBTEST.COM
Aug 11 19:12:02 dist-test-slave-t7g5 krb5kdc[14712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475522, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.61.129@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM
I20260811 19:12:02.772573 16363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:02.726850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39548 (local address 127.3.61.130:32931)
0811 19:12:02.738541 (+ 11691us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:02.738554 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:02.740078 (+ 1524us) server_negotiation.cc:407] Connection header received
0811 19:12:02.740335 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:02.740346 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:02.740743 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:02.740932 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:02.744089 (+ 3157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:02.746058 (+ 1969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:02.748713 (+ 2655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:02.749910 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:02.750041 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:02.758744 (+ 8703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:02.758768 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:02.758789 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:02.758857 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:02.763889 (+ 5032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:02.765528 (+ 1639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:02.765539 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:02.765543 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:02.766130 (+ 587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:02.767986 (+ 1856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:02.767992 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:02.767996 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:02.769363 (+ 1367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:02.769773 (+ 410us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:02.771869 (+ 2096us) server_negotiation.cc:299] Negotiation successful
0811 19:12:02.772161 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11494,"thread_start_us":11401,"threads_started":1}
I20260811 19:12:02.774116 15295 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:12:02.780068 16146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:02.727718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:59293 (local address 127.3.61.188:46509)
0811 19:12:02.727940 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:02.727957 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:02.743215 (+ 15258us) server_negotiation.cc:407] Connection header received
0811 19:12:02.748821 (+ 5606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:02.748829 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:02.748950 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:02.749036 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:02.756717 (+ 7681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:02.758026 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:02.761832 (+ 3806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:02.762595 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:02.762710 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:02.769688 (+ 6978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:02.769698 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:02.769703 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:02.769750 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:02.774522 (+ 4772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:02.775675 (+ 1153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:02.775682 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:02.775686 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:02.775926 (+ 240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:02.776961 (+ 1035us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:02.776967 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:02.776970 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:02.777576 (+ 606us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:02.777953 (+ 377us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:02.779300 (+ 1347us) server_negotiation.cc:299] Negotiation successful
0811 19:12:02.779650 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 19:12:02.782521 16368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:02.727356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:49669 (local address 127.3.61.189:43587)
0811 19:12:02.745095 (+ 17739us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:02.745102 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:02.750247 (+ 5145us) server_negotiation.cc:407] Connection header received
0811 19:12:02.753218 (+ 2971us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:02.753224 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:02.753348 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:02.753453 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:02.756714 (+ 3261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:02.758144 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:02.761860 (+ 3716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:02.762771 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:02.762890 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:02.770600 (+ 7710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:02.770606 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:02.770609 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:02.770654 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:02.775313 (+ 4659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:02.779007 (+ 3694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:02.779014 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:02.779018 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:02.779214 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:02.779708 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:02.779714 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:02.779717 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:02.780083 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:02.780238 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:02.781871 (+ 1633us) server_negotiation.cc:299] Negotiation successful
0811 19:12:02.782123 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17528,"thread_start_us":18564,"threads_started":1}
I20260811 19:12:02.783964 16364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:02.726777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.189:43587 (local address 127.3.61.129:49669)
0811 19:12:02.746668 (+ 19891us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:02.746691 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:02.753006 (+ 6315us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:02.754098 (+ 1092us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:02.754102 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:02.755151 (+ 1049us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:02.756433 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:02.756476 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:02.758372 (+ 1896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:02.758377 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:02.761557 (+ 3180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:02.761575 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:02.763394 (+ 1819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:02.763404 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:02.765609 (+ 2205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:02.766457 (+ 848us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:02.766483 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:02.770339 (+ 3856us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:12:02.777806 (+ 7467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:02.777811 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:02.777816 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:02.778536 (+ 720us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:02.779388 (+ 852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:02.779393 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:02.779397 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:02.779570 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:02.780299 (+ 729us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:12:02.780304 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:12:02.780477 (+ 173us) client_negotiation.cc:769] Sending connection context
0811 19:12:02.780658 (+ 181us) client_negotiation.cc:240] Negotiation successful
0811 19:12:02.781469 (+ 811us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19655,"mutex_wait_us":1425,"thread_start_us":14187,"threads_started":1}
I20260811 19:12:02.783922 16369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:02.726777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:46509 (local address 127.3.61.129:59293)
0811 19:12:02.742936 (+ 16159us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:02.743019 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:02.748370 (+ 5351us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:02.749784 (+ 1414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:02.749801 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:02.750982 (+ 1181us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:02.754747 (+ 3765us) spinlock_profiling.cc:243] Waited 1.9 ms on lock 0x61200002c628. stack: 00007f31e9c7b66e 00007f31e9c7b430 00007f31e8d025ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f31e9c710fd 00007f31e9c70d1e 00007f31fcb7a3cd 00007f31fcb7a36c 00007f31eadaa757 00007f31f067f03a 00007f31f0706dca 00007f31f0704f68 00007f31f073882f 00007f31f073858d
0811 19:12:02.756437 (+ 1690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:02.756472 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:02.758537 (+ 2065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:02.758548 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:02.761535 (+ 2987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:02.761559 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:02.763026 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:02.763032 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:02.763450 (+ 418us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:02.764511 (+ 1061us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:02.764546 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:02.769223 (+ 4677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:12:02.774751 (+ 5528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:02.774762 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:02.774782 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:02.775525 (+ 743us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:02.776106 (+ 581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:02.776111 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:02.776115 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:02.776637 (+ 522us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:02.778086 (+ 1449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:12:02.778096 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:12:02.778708 (+ 612us) client_negotiation.cc:769] Sending connection context
0811 19:12:02.780938 (+ 2230us) client_negotiation.cc:240] Negotiation successful
0811 19:12:02.781747 (+ 809us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15637,"mutex_wait_us":425,"spinlock_wait_cycles":4381440,"thread_start_us":19139,"threads_started":1}
I20260811 19:12:02.785959 16370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:02.746979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33527 (local address 127.3.61.129:60935)
0811 19:12:02.753962 (+ 6983us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:02.753986 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:02.754105 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:02.758586 (+ 4481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:02.758591 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:02.759535 (+ 944us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:02.761112 (+ 1577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:02.761146 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:02.763241 (+ 2095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:02.763246 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:02.765507 (+ 2261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:02.765540 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:02.767306 (+ 1766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:02.767314 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:02.767561 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:02.768500 (+ 939us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:02.768524 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:02.776220 (+ 7696us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:12:02.781999 (+ 5779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:02.782005 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:02.782009 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:02.782773 (+ 764us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:02.783494 (+ 721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:02.783505 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:02.783509 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:02.783795 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:02.784663 (+ 868us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:12:02.784668 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:12:02.784878 (+ 210us) client_negotiation.cc:769] Sending connection context
0811 19:12:02.785036 (+ 158us) client_negotiation.cc:240] Negotiation successful
0811 19:12:02.785411 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6787,"thread_start_us":7198,"threads_started":1}
I20260811 19:12:02.786507 16359 heartbeater.cc:347] Connected to a master server at 127.3.61.188:46509
I20260811 19:12:02.786605 16360 heartbeater.cc:347] Connected to a master server at 127.3.61.189:43587
I20260811 19:12:02.787042 16360 heartbeater.cc:464] Registering TS with master...
I20260811 19:12:02.787804 16371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:02.747007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:60935 (local address 127.3.61.190:33527)
0811 19:12:02.758044 (+ 11037us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:02.758050 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:02.758070 (+ 20us) server_negotiation.cc:407] Connection header received
0811 19:12:02.758135 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:02.758139 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:02.758226 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:02.758329 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:02.761484 (+ 3155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:02.763022 (+ 1538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:02.765775 (+ 2753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:02.766768 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:02.766936 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:02.776834 (+ 9898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:02.776849 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:02.776855 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:02.776934 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:02.781795 (+ 4861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:02.782993 (+ 1198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:02.782999 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:02.783003 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:02.783327 (+ 324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:02.783955 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:02.783960 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:02.783963 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:02.784426 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:02.784583 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:02.786323 (+ 1740us) server_negotiation.cc:299] Negotiation successful
0811 19:12:02.787387 (+ 1064us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10881,"thread_start_us":12696,"threads_started":1}
I20260811 19:12:02.788328 16360 heartbeater.cc:511] Master 127.3.61.189:43587 requested a full tablet report, sending...
I20260811 19:12:02.788971 16358 heartbeater.cc:347] Connected to a master server at 127.3.61.190:33527
I20260811 19:12:02.789152 16358 heartbeater.cc:464] Registering TS with master...
I20260811 19:12:02.787056 16359 heartbeater.cc:464] Registering TS with master...
I20260811 19:12:02.789902 16358 heartbeater.cc:511] Master 127.3.61.190:33527 requested a full tablet report, sending...
I20260811 19:12:02.790108 16359 heartbeater.cc:511] Master 127.3.61.188:46509 requested a full tablet report, sending...
I20260811 19:12:02.791317 14792 ts_manager.cc:194] Re-registered known tserver with Master: 3e21de71b6e9439f80d484412bed0c8a (127.3.61.129:38449)
I20260811 19:12:02.791529 15763 ts_manager.cc:194] Registered new tserver with Master: 3e21de71b6e9439f80d484412bed0c8a (127.3.61.129:38449)
I20260811 19:12:02.792124 16066 ts_manager.cc:194] Registered new tserver with Master: 3e21de71b6e9439f80d484412bed0c8a (127.3.61.129:38449)
I20260811 19:12:02.792559 14792 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.61.129@KRBTEST.COM'} at 127.3.61.129:60935
I20260811 19:12:02.869052 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 15100
I20260811 19:12:02.898274 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.130:32931
--local_ip_for_outbound_sockets=127.3.61.130
--tserver_master_addrs=127.3.61.190:33527
--webserver_port=46603
--webserver_interface=127.3.61.130
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/krb5kdc/oryx_127.3.61.130.keytab
--principal=oryx/127.3.61.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:34789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.3.61.190:33527,127.3.61.189:43587
--tserver_master_addrs=127.3.61.190:33527,127.3.61.189:43587,127.3.61.188:46509 with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:12:03.295428 16373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:12:03.295898 16373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:12:03.296075 16373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:12:03.306015 16373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:12:03.306185 16373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:12:03.306349 16373 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.130
I20260811 19:12:03.317797 16373 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:34789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/krb5kdc/oryx_127.3.61.130.keytab
--principal=oryx/127.3.61.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.130:32931
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.61.130
--webserver_port=46603
--tserver_master_addrs=127.3.61.190:33527,127.3.61.189:43587,127.3.61.188:46509
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.130
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:12:03.319372 16373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:12:03.321107 16373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:12:03.338234 16379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:03.338668 16380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:03.338872 16382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:12:03.340163 16373 server_base.cc:1034] running on GCE node
I20260811 19:12:03.340988 16373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:12:03.342417 16373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:12:03.343797 16373 hybrid_clock.cc:648] HybridClock initialized: now 1786475523343710 us; error 85 us; skew 500 ppm
I20260811 19:12:03.344244 16373 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:12:03 dist-test-slave-t7g5 krb5kdc[14712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475523, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.61.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:12:03.348101 16373 init.cc:377] Logged in from keytab as oryx/127.3.61.130@KRBTEST.COM (short username oryx)
I20260811 19:12:03.349989 16373 webserver.cc:460] Webserver started at http://127.3.61.130:46603/ using document root <none> and password file <none>
I20260811 19:12:03.350814 16373 fs_manager.cc:359] Metadata directory not provided
I20260811 19:12:03.350953 16373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:12:03.357602 16373 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.002s
I20260811 19:12:03.361840 16395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:03.363945 16373 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260811 19:12:03.364184 16373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-1/wal
uuid: "06c375ae44c0466c8b91db22d0b0765c"
format_stamp: "Formatted at 2026-08-11 19:11:59 on dist-test-slave-t7g5"
server_key: "a383bed5a9214effd78e77ef03c84f85"
server_key_iv: "efd1b422c5b65fdf6b413f6fc1beece2"
server_key_version: "encryptionkey@0"
I20260811 19:12:03.365846 16373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:12:03.387454 16373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:12:03.392947 16373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:12:03.393268 16373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:12:03.394490 16373 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:12:03.396128 16373 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:12:03.396231 16373 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:03.396342 16373 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:12:03.396420 16373 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:03.434648 16373 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.130:32931
I20260811 19:12:03.434734 16579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.130:32931 every 8 connection(s)
I20260811 19:12:03.436743 16373 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
I20260811 19:12:03.441993 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 16373
I20260811 19:12:03.442322 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 15308
Aug 11 19:12:03 dist-test-slave-t7g5 krb5kdc[14712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475523, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.61.130@KRBTEST.COM for oryx/127.3.61.189@KRBTEST.COM
Aug 11 19:12:03 dist-test-slave-t7g5 krb5kdc[14712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475523, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.61.130@KRBTEST.COM for oryx/127.3.61.188@KRBTEST.COM
Aug 11 19:12:03 dist-test-slave-t7g5 krb5kdc[14712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475523, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.61.130@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM
I20260811 19:12:03.477119 16590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:03.440527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:56271 (local address 127.3.61.189:43587)
0811 19:12:03.441442 (+ 915us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:03.441448 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:03.450461 (+ 9013us) server_negotiation.cc:407] Connection header received
0811 19:12:03.451651 (+ 1190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:03.451659 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:03.451768 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:03.451910 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:03.455116 (+ 3206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:03.456360 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:03.461299 (+ 4939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:03.461968 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:03.462072 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:03.468565 (+ 6493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:03.468573 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:03.468578 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:03.468622 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:03.472270 (+ 3648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:03.473470 (+ 1200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:03.473476 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:03.473480 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:03.473695 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:03.474482 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:03.474488 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:03.474491 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:03.474944 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:03.475078 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:03.476343 (+ 1265us) server_negotiation.cc:299] Negotiation successful
0811 19:12:03.476649 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":778,"thread_start_us":666,"threads_started":1}
I20260811 19:12:03.479974 16593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:03.441240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.189:43587 (local address 127.3.61.130:56271)
0811 19:12:03.450227 (+ 8987us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:03.450265 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:03.450970 (+ 705us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:03.452506 (+ 1536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:03.452523 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:03.453701 (+ 1178us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:03.454720 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:03.454762 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:03.457307 (+ 2545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:03.457313 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:03.460486 (+ 3173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:03.460500 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:03.462503 (+ 2003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:03.462512 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:03.462887 (+ 375us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:03.463735 (+ 848us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:03.463775 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:03.467718 (+ 3943us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:12:03.472511 (+ 4793us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:03.472521 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:03.472540 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:03.473314 (+ 774us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:03.473847 (+ 533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:03.473852 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:03.473856 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:03.474246 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:03.475198 (+ 952us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:12:03.475208 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:12:03.475759 (+ 551us) client_negotiation.cc:769] Sending connection context
0811 19:12:03.477465 (+ 1706us) client_negotiation.cc:240] Negotiation successful
0811 19:12:03.478197 (+ 732us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2462,"mutex_wait_us":1,"thread_start_us":580,"threads_started":1}
I20260811 19:12:03.481021 16597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:03.444244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:47683 (local address 127.3.61.190:33527)
0811 19:12:03.454743 (+ 10499us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:03.454749 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:03.459917 (+ 5168us) server_negotiation.cc:407] Connection header received
0811 19:12:03.460374 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:03.460380 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:03.460459 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:03.460587 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:03.462742 (+ 2155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:03.463937 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:03.466134 (+ 2197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:03.467066 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:03.467159 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:03.472728 (+ 5569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:03.472736 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:03.472740 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:03.472789 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:03.477494 (+ 4705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:03.478480 (+ 986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:03.478485 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:03.478488 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:03.478711 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:03.479229 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:03.479235 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:03.479238 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:03.479601 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:03.479738 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:03.480359 (+ 621us) server_negotiation.cc:299] Negotiation successful
0811 19:12:03.480606 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10357,"thread_start_us":10670,"threads_started":1}
Aug 11 19:12:03 dist-test-slave-t7g5 krb5kdc[14712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475517, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.61.131@KRBTEST.COM
I20260811 19:12:03.482234 16585 heartbeater.cc:347] Connected to a master server at 127.3.61.189:43587
I20260811 19:12:03.482901 16585 heartbeater.cc:464] Registering TS with master...
I20260811 19:12:03.483105 16592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:03.441836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:44813 (local address 127.3.61.188:46509)
0811 19:12:03.442471 (+ 635us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:03.442480 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:03.455317 (+ 12837us) server_negotiation.cc:407] Connection header received
0811 19:12:03.455413 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:03.455420 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:03.455541 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:03.455734 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:03.457750 (+ 2016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:03.460475 (+ 2725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:03.466013 (+ 5538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:03.466732 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:03.466901 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:03.473789 (+ 6888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:03.473803 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:03.473809 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:03.473860 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:03.477342 (+ 3482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:03.478326 (+ 984us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:03.478333 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:03.478337 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:03.478501 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:03.479005 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:03.479011 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:03.479014 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:03.479548 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:03.479767 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:03.482325 (+ 2558us) server_negotiation.cc:299] Negotiation successful
0811 19:12:03.482619 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":1115,"threads_started":1}
I20260811 19:12:03.483944 16591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:03.442778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:46509 (local address 127.3.61.130:44813)
0811 19:12:03.445047 (+ 2269us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:03.445156 (+ 109us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:03.450970 (+ 5814us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:03.455980 (+ 5010us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:03.455986 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:03.456596 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:03.456908 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:03.456922 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:03.460914 (+ 3992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:03.460922 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:03.465789 (+ 4867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:03.465807 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:03.467339 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:03.467346 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:03.467537 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:03.468320 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:03.468347 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:03.472237 (+ 3890us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:12:03.477511 (+ 5274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:03.477516 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:03.477519 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:03.478161 (+ 642us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:03.478644 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:03.478649 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:03.478653 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:03.478845 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:03.479853 (+ 1008us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:12:03.479859 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:12:03.480066 (+ 207us) client_negotiation.cc:769] Sending connection context
0811 19:12:03.480209 (+ 143us) client_negotiation.cc:240] Negotiation successful
0811 19:12:03.481045 (+ 836us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2055,"mutex_wait_us":2812,"thread_start_us":810,"threads_started":1}
I20260811 19:12:03.484040 16594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:03.444108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33527 (local address 127.3.61.130:47683)
0811 19:12:03.459767 (+ 15659us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:03.459797 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:03.460205 (+ 408us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:03.460834 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:03.460839 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:03.461838 (+ 999us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:03.462567 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:03.462580 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:03.464127 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:03.464132 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:03.465980 (+ 1848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:03.465993 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:03.467513 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:03.467521 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:03.467767 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:03.468468 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:03.468496 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:03.472497 (+ 4001us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:12:03.477672 (+ 5175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:03.477678 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:03.477682 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:03.478339 (+ 657us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:03.478887 (+ 548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:03.478893 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:03.478897 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:03.479095 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:03.479819 (+ 724us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:12:03.479824 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:12:03.480016 (+ 192us) client_negotiation.cc:769] Sending connection context
0811 19:12:03.481222 (+ 1206us) client_negotiation.cc:240] Negotiation successful
0811 19:12:03.481553 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15507,"mutex_wait_us":1,"spinlock_wait_cycles":13824,"thread_start_us":15832,"threads_started":1}
I20260811 19:12:03.484470 16585 heartbeater.cc:511] Master 127.3.61.189:43587 requested a full tablet report, sending...
I20260811 19:12:03.485409 16584 heartbeater.cc:347] Connected to a master server at 127.3.61.188:46509
I20260811 19:12:03.485636 16584 heartbeater.cc:464] Registering TS with master...
I20260811 19:12:03.486373 16584 heartbeater.cc:511] Master 127.3.61.188:46509 requested a full tablet report, sending...
I20260811 19:12:03.486431 16580 heartbeater.cc:347] Connected to a master server at 127.3.61.190:33527
I20260811 19:12:03.486632 16580 heartbeater.cc:464] Registering TS with master...
I20260811 19:12:03.486953 15763 ts_manager.cc:194] Registered new tserver with Master: 06c375ae44c0466c8b91db22d0b0765c (127.3.61.130:32931)
I20260811 19:12:03.487232 16580 heartbeater.cc:511] Master 127.3.61.190:33527 requested a full tablet report, sending...
I20260811 19:12:03.487810 16066 ts_manager.cc:194] Registered new tserver with Master: 06c375ae44c0466c8b91db22d0b0765c (127.3.61.130:32931)
I20260811 19:12:03.488444 14792 ts_manager.cc:194] Re-registered known tserver with Master: 06c375ae44c0466c8b91db22d0b0765c (127.3.61.130:32931)
I20260811 19:12:03.489759 14792 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.61.130@KRBTEST.COM'} at 127.3.61.130:47683
I20260811 19:12:03.495505 16601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:03.449808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39884 (local address 127.3.61.131:34125)
0811 19:12:03.461172 (+ 11364us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:03.461183 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:03.462281 (+ 1098us) server_negotiation.cc:407] Connection header received
0811 19:12:03.462391 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:03.462402 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:03.462801 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:03.463040 (+ 239us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:03.466712 (+ 3672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:03.468907 (+ 2195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:03.472871 (+ 3964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:03.474079 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:03.474231 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:03.485753 (+ 11522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:03.485781 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:03.485800 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:03.485848 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:03.491448 (+ 5600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:03.492805 (+ 1357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:03.492813 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:03.492816 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:03.493026 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:03.493549 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:03.493553 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:03.493556 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:03.494016 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:03.494396 (+ 380us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:03.494707 (+ 311us) server_negotiation.cc:299] Negotiation successful
0811 19:12:03.495018 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11139,"thread_start_us":10967,"threads_started":1}
I20260811 19:12:03.497622 15503 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:12:03.593706 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 15308
I20260811 19:12:03.621930 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.131:34125
--local_ip_for_outbound_sockets=127.3.61.131
--tserver_master_addrs=127.3.61.190:33527
--webserver_port=33447
--webserver_interface=127.3.61.131
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/krb5kdc/oryx_127.3.61.131.keytab
--principal=oryx/127.3.61.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:34789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.3.61.190:33527,127.3.61.189:43587
--tserver_master_addrs=127.3.61.190:33527,127.3.61.189:43587,127.3.61.188:46509 with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:12:03.795409 16358 heartbeater.cc:503] Master 127.3.61.190:33527 was elected leader, sending a full tablet report...
W20260811 19:12:04.018224 16605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:12:04.018610 16605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:12:04.018728 16605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:12:04.026513 16605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:12:04.026678 16605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:12:04.026801 16605 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.131
I20260811 19:12:04.039055 16605 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:34789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/krb5kdc/oryx_127.3.61.131.keytab
--principal=oryx/127.3.61.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.131:34125
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.61.131
--webserver_port=33447
--tserver_master_addrs=127.3.61.190:33527,127.3.61.189:43587,127.3.61.188:46509
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.131
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:12:04.040568 16605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:12:04.042032 16605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:12:04.057301 16612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:04.057281 16615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:04.057339 16613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:12:04.058305 16605 server_base.cc:1034] running on GCE node
I20260811 19:12:04.059016 16605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:12:04.060175 16605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:12:04.061619 16605 hybrid_clock.cc:648] HybridClock initialized: now 1786475524061534 us; error 83 us; skew 500 ppm
I20260811 19:12:04.062049 16605 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:12:04 dist-test-slave-t7g5 krb5kdc[14712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475524, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.61.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:12:04.065526 16605 init.cc:377] Logged in from keytab as oryx/127.3.61.131@KRBTEST.COM (short username oryx)
I20260811 19:12:04.067343 16605 webserver.cc:460] Webserver started at http://127.3.61.131:33447/ using document root <none> and password file <none>
I20260811 19:12:04.068123 16605 fs_manager.cc:359] Metadata directory not provided
I20260811 19:12:04.068253 16605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:12:04.074472 16605 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 19:12:04.077837 16624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:04.079562 16605 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260811 19:12:04.079764 16605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-2/wal
uuid: "9117d1dafdbe4e6f804a6f7f742596b6"
format_stamp: "Formatted at 2026-08-11 19:12:00 on dist-test-slave-t7g5"
server_key: "4678e81847ce83504b3bf729f70388c5"
server_key_iv: "271ae640c3f2b54d970a228a6a2d37e3"
server_key_version: "encryptionkey@0"
I20260811 19:12:04.080876 16605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:12:04.098479 16605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:12:04.105674 16605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:12:04.105931 16605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:12:04.107703 16605 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:12:04.109306 16605 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:12:04.109417 16605 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:04.109504 16605 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:12:04.109563 16605 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:04.149470 16605 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.131:34125
I20260811 19:12:04.149564 16819 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.131:34125 every 8 connection(s)
I20260811 19:12:04.151407 16605 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
I20260811 19:12:04.158666 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 16605
Aug 11 19:12:04 dist-test-slave-t7g5 krb5kdc[14712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475524, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.61.131@KRBTEST.COM for oryx/127.3.61.189@KRBTEST.COM
Aug 11 19:12:04 dist-test-slave-t7g5 krb5kdc[14712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475524, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.61.131@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM
Aug 11 19:12:04 dist-test-slave-t7g5 krb5kdc[14712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475524, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.61.131@KRBTEST.COM for oryx/127.3.61.188@KRBTEST.COM
I20260811 19:12:04.219910 16832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:04.155465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.189:43587 (local address 127.3.61.131:51281)
0811 19:12:04.177877 (+ 22412us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:04.177906 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:04.178035 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:04.184759 (+ 6724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:04.184765 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:04.185587 (+ 822us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:04.185978 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:04.185991 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:04.191729 (+ 5738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:04.191735 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:04.193337 (+ 1602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:04.193354 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:04.194990 (+ 1636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:04.194997 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:04.195183 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:04.195856 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:04.195895 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:04.201477 (+ 5582us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:12:04.214786 (+ 13309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:04.214792 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:04.214795 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:04.215567 (+ 772us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:04.216564 (+ 997us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:04.216568 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:04.216571 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:04.216871 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:04.217592 (+ 721us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:12:04.217597 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:12:04.217838 (+ 241us) client_negotiation.cc:769] Sending connection context
0811 19:12:04.218002 (+ 164us) client_negotiation.cc:240] Negotiation successful
0811 19:12:04.218246 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17176,"mutex_wait_us":4,"spinlock_wait_cycles":10624,"thread_start_us":16780,"threads_started":1}
I20260811 19:12:04.220160 16828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:04.154981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:56985 (local address 127.3.61.190:33527)
0811 19:12:04.182342 (+ 27361us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:04.182349 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:04.182373 (+ 24us) server_negotiation.cc:407] Connection header received
0811 19:12:04.182461 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:04.182468 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:04.182559 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:04.182679 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:04.186224 (+ 3545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:04.187488 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:04.191007 (+ 3519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:04.193735 (+ 2728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:04.193878 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:04.204117 (+ 10239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:04.204127 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:04.204134 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:04.204212 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:04.210225 (+ 6013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:04.213451 (+ 3226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:04.213459 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:04.213463 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:04.213800 (+ 337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:04.214771 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:04.214778 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:04.214782 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:04.215656 (+ 874us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:04.215926 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:04.218983 (+ 3057us) server_negotiation.cc:299] Negotiation successful
0811 19:12:04.219284 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27222,"thread_start_us":27092,"threads_started":1}
I20260811 19:12:04.220846 16833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:04.155465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33527 (local address 127.3.61.131:56985)
0811 19:12:04.174433 (+ 18968us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:04.174512 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:04.177323 (+ 2811us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:04.183076 (+ 5753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:04.183092 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:04.184075 (+ 983us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:04.185170 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:04.185205 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:04.187829 (+ 2624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:04.187837 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:04.190697 (+ 2860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:04.190732 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:04.194069 (+ 3337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:04.194081 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:04.194452 (+ 371us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:04.195928 (+ 1476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:04.195950 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:04.203769 (+ 7819us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:12:04.210709 (+ 6940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:04.210726 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:04.210753 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:04.211751 (+ 998us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:04.213985 (+ 2234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:04.213992 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:04.213996 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:04.214510 (+ 514us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:04.216497 (+ 1987us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:12:04.216508 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:12:04.217089 (+ 581us) client_negotiation.cc:769] Sending connection context
0811 19:12:04.217463 (+ 374us) client_negotiation.cc:240] Negotiation successful
0811 19:12:04.218154 (+ 691us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17741,"mutex_wait_us":41,"spinlock_wait_cycles":1658368,"thread_start_us":17364,"threads_started":1}
I20260811 19:12:04.222249 16835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:04.174137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:46509 (local address 127.3.61.131:38377)
0811 19:12:04.181625 (+ 7488us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:04.181679 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:04.181825 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:04.184592 (+ 2767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:04.184596 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:04.185313 (+ 717us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:04.185582 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:04.185591 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:04.192649 (+ 7058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:04.192655 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:04.195630 (+ 2975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:04.195646 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:04.197292 (+ 1646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:04.197301 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:04.197528 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:04.198574 (+ 1046us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:04.198600 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:04.204352 (+ 5752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:12:04.215707 (+ 11355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:04.215712 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:04.215716 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:04.216312 (+ 596us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:04.220169 (+ 3857us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:04.220174 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:04.220177 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:04.220336 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:04.221179 (+ 843us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:12:04.221184 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:12:04.221389 (+ 205us) client_negotiation.cc:769] Sending connection context
0811 19:12:04.221560 (+ 171us) client_negotiation.cc:240] Negotiation successful
0811 19:12:04.221889 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7341,"thread_start_us":7112,"threads_started":1}
I20260811 19:12:04.222997 16822 heartbeater.cc:347] Connected to a master server at 127.3.61.190:33527
I20260811 19:12:04.223388 16822 heartbeater.cc:464] Registering TS with master...
I20260811 19:12:04.223951 16836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:04.174246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:38377 (local address 127.3.61.188:46509)
0811 19:12:04.184228 (+ 9982us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:04.184235 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:04.184254 (+ 19us) server_negotiation.cc:407] Connection header received
0811 19:12:04.184317 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:04.184323 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:04.184414 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:04.184535 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:04.188941 (+ 4406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:04.192331 (+ 3390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:04.196200 (+ 3869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:04.197022 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:04.197772 (+ 750us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:04.204640 (+ 6868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:04.204649 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:04.204654 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:04.204704 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:04.215511 (+ 10807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:04.219816 (+ 4305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:04.219823 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:04.219826 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:04.220020 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:04.220509 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:04.220514 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:04.220517 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:04.220901 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:04.223097 (+ 2196us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:04.223229 (+ 132us) server_negotiation.cc:299] Negotiation successful
0811 19:12:04.223495 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9844,"thread_start_us":9735,"threads_started":1}
I20260811 19:12:04.223970 16826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:04.154865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:51281 (local address 127.3.61.189:43587)
0811 19:12:04.183530 (+ 28665us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:04.183537 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:04.183561 (+ 24us) server_negotiation.cc:407] Connection header received
0811 19:12:04.183630 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:04.183635 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:04.183716 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:04.187722 (+ 4006us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:04.187937 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:04.191520 (+ 3583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:04.194082 (+ 2562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:04.194740 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:04.195556 (+ 816us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:04.202211 (+ 6655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:04.202219 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:04.202224 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:04.202402 (+ 178us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:04.210312 (+ 7910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:04.215768 (+ 5456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:04.215774 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:04.215778 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:04.216004 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:04.217051 (+ 1047us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:04.217056 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:04.217059 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:04.217461 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:04.223047 (+ 5586us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:04.223219 (+ 172us) server_negotiation.cc:299] Negotiation successful
0811 19:12:04.223513 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28498,"thread_start_us":29778,"threads_started":1}
I20260811 19:12:04.224546 16822 heartbeater.cc:511] Master 127.3.61.190:33527 requested a full tablet report, sending...
I20260811 19:12:04.224948 16824 heartbeater.cc:347] Connected to a master server at 127.3.61.189:43587
I20260811 19:12:04.225118 16824 heartbeater.cc:464] Registering TS with master...
I20260811 19:12:04.225816 16824 heartbeater.cc:511] Master 127.3.61.189:43587 requested a full tablet report, sending...
I20260811 19:12:04.227051 14792 ts_manager.cc:194] Re-registered known tserver with Master: 9117d1dafdbe4e6f804a6f7f742596b6 (127.3.61.131:34125)
I20260811 19:12:04.227535 15763 ts_manager.cc:194] Registered new tserver with Master: 9117d1dafdbe4e6f804a6f7f742596b6 (127.3.61.131:34125)
I20260811 19:12:04.228233 14792 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.61.131@KRBTEST.COM'} at 127.3.61.131:56985
I20260811 19:12:04.232429 16823 heartbeater.cc:347] Connected to a master server at 127.3.61.188:46509
I20260811 19:12:04.232630 16823 heartbeater.cc:464] Registering TS with master...
I20260811 19:12:04.233227 16823 heartbeater.cc:511] Master 127.3.61.188:46509 requested a full tablet report, sending...
I20260811 19:12:04.235033 16066 ts_manager.cc:194] Registered new tserver with Master: 9117d1dafdbe4e6f804a6f7f742596b6 (127.3.61.131:34125)
I20260811 19:12:04.240661 16848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:04.193463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47352 (local address 127.3.61.189:43587)
0811 19:12:04.203639 (+ 10176us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:04.203649 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:04.203700 (+ 51us) server_negotiation.cc:407] Connection header received
0811 19:12:04.203822 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:04.203834 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:04.206613 (+ 2779us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:04.207690 (+ 1077us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:04.213392 (+ 5702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:04.216404 (+ 3012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:04.221595 (+ 5191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:04.222351 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:04.222465 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:04.229782 (+ 7317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:04.229792 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:04.229797 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:04.229871 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:04.236702 (+ 6831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:04.237858 (+ 1156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:04.237863 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:04.237866 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:04.238144 (+ 278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:04.238674 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:04.238681 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:04.238684 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:04.239103 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:04.239368 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:04.239837 (+ 469us) server_negotiation.cc:299] Negotiation successful
0811 19:12:04.240148 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9893,"thread_start_us":9178,"threads_started":1}
I20260811 19:12:04.246218 16849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:04.200864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48390 (local address 127.3.61.190:33527)
0811 19:12:04.216813 (+ 15949us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:04.216818 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:04.216837 (+ 19us) server_negotiation.cc:407] Connection header received
0811 19:12:04.216906 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:04.216911 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:04.216987 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:04.217094 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:04.220872 (+ 3778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:04.222317 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:04.231794 (+ 9477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:04.232548 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:04.232681 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:04.237838 (+ 5157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:04.237845 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:04.237850 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:04.237899 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:04.242399 (+ 4500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:04.243320 (+ 921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:04.243324 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:04.243327 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:04.243502 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:04.243931 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:04.243935 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:04.243937 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:04.244409 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:04.244708 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:04.245272 (+ 564us) server_negotiation.cc:299] Negotiation successful
0811 19:12:04.245685 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15797,"thread_start_us":15626,"threads_started":1}
I20260811 19:12:04.249070 16836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:04.223517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34006 (local address 127.3.61.188:46509)
0811 19:12:04.224537 (+ 1020us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:04.224544 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:04.227642 (+ 3098us) server_negotiation.cc:407] Connection header received
0811 19:12:04.227708 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:04.227713 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:04.227793 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:04.227894 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:04.231485 (+ 3591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:04.232990 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:04.235663 (+ 2673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:04.236480 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:04.236637 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:04.241014 (+ 4377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:04.241023 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:04.241028 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:04.241098 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:04.245291 (+ 4193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:04.246464 (+ 1173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:04.246471 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:04.246474 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:04.246692 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:04.247179 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:04.247184 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:04.247187 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:04.247645 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:04.247953 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:04.248408 (+ 455us) server_negotiation.cc:299] Negotiation successful
0811 19:12:04.248689 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":896,"thread_start_us":7962,"threads_started":1}
I20260811 19:12:04.252004 14792 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48390:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 19:12:04.261081 14792 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:12:04.316480 16882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:04.285161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38770 (local address 127.3.61.129:38449)
0811 19:12:04.303729 (+ 18568us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:04.303740 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:04.305178 (+ 1438us) server_negotiation.cc:407] Connection header received
0811 19:12:04.305443 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:04.305454 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:04.305857 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:04.306126 (+ 269us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:12:04.307328 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:04.309324 (+ 1996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:04.312141 (+ 2817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:04.314145 (+ 2004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:04.314409 (+ 264us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:04.314675 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:04.315875 (+ 1200us) server_negotiation.cc:299] Negotiation successful
0811 19:12:04.316084 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18320,"thread_start_us":18583,"threads_started":1}
I20260811 19:12:04.317803 16876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:04.284381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:38449 (local address 127.0.0.1:38770)
0811 19:12:04.305022 (+ 20641us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:04.305044 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:04.305233 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:04.306416 (+ 1183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:04.306422 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:04.306452 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:12:04.307019 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:04.307033 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:04.309477 (+ 2444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:04.309483 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:04.311951 (+ 2468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:04.311970 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:04.314735 (+ 2765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:04.314745 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:04.314948 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:04.314989 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 19:12:04.315182 (+ 193us) client_negotiation.cc:240] Negotiation successful
0811 19:12:04.315366 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20486,"mutex_wait_us":1,"thread_start_us":22275,"threads_started":1}
I20260811 19:12:04.318478 16881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:04.284102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.130:32931 (local address 127.0.0.1:34444)
0811 19:12:04.301303 (+ 17201us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:04.301331 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:04.301555 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:04.307463 (+ 5908us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:04.307467 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:04.307487 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:12:04.308219 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:04.308232 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:04.309900 (+ 1668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:04.309905 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:04.314806 (+ 4901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:04.314825 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:04.317142 (+ 2317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:04.317149 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:04.317621 (+ 472us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:04.317703 (+ 82us) client_negotiation.cc:769] Sending connection context
0811 19:12:04.317817 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 19:12:04.317966 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16921,"mutex_wait_us":1,"thread_start_us":20346,"threads_started":1}
I20260811 19:12:04.319088 16883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:04.285612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36888 (local address 127.3.61.131:34125)
0811 19:12:04.304068 (+ 18456us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:04.304076 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:04.305738 (+ 1662us) server_negotiation.cc:407] Connection header received
0811 19:12:04.307327 (+ 1589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:04.307336 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:04.307543 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:04.307768 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:12:04.309125 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:04.311107 (+ 1982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:04.315206 (+ 4099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:04.317113 (+ 1907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:04.317258 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:04.317545 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:04.318208 (+ 663us) server_negotiation.cc:299] Negotiation successful
0811 19:12:04.318393 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18209,"thread_start_us":17912,"threads_started":1}
I20260811 19:12:04.319121 16880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:04.284753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34444 (local address 127.3.61.130:32931)
0811 19:12:04.306811 (+ 22058us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:04.306821 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:04.306852 (+ 31us) server_negotiation.cc:407] Connection header received
0811 19:12:04.306960 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:04.306968 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:04.307246 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:04.307469 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:12:04.308419 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:04.309761 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:04.315070 (+ 5309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:04.316828 (+ 1758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:04.316969 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:04.317191 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:04.318687 (+ 1496us) server_negotiation.cc:299] Negotiation successful
0811 19:12:04.318861 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21830,"thread_start_us":22848,"threads_started":1}
I20260811 19:12:04.318544 16878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:04.285020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.131:34125 (local address 127.0.0.1:36888)
0811 19:12:04.305540 (+ 20520us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:04.305558 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:04.307079 (+ 1521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:04.308148 (+ 1069us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:04.308152 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:04.308177 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:12:04.308834 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:04.308850 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:04.312143 (+ 3293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:04.312147 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:04.314575 (+ 2428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:04.314593 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:04.317506 (+ 2913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:04.317512 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:04.317907 (+ 395us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:04.317939 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 19:12:04.318020 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 19:12:04.318149 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20362,"thread_start_us":15896,"threads_started":1}
I20260811 19:12:04.322970 16246 tablet_service.cc:1467] Processing CreateTablet for tablet 6709e98d474f4acd9ceb70b0b7f74237 (DEFAULT_TABLE table=test-table [id=802b55ec40874b53b0b1f03ae854a4bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:12:04.323876 16473 tablet_service.cc:1467] Processing CreateTablet for tablet 6709e98d474f4acd9ceb70b0b7f74237 (DEFAULT_TABLE table=test-table [id=802b55ec40874b53b0b1f03ae854a4bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:12:04.323876 16712 tablet_service.cc:1467] Processing CreateTablet for tablet 6709e98d474f4acd9ceb70b0b7f74237 (DEFAULT_TABLE table=test-table [id=802b55ec40874b53b0b1f03ae854a4bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:12:04.325016 16246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6709e98d474f4acd9ceb70b0b7f74237. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:12:04.325470 16473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6709e98d474f4acd9ceb70b0b7f74237. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:12:04.325469 16712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6709e98d474f4acd9ceb70b0b7f74237. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:12:04.356926 16887 tablet_bootstrap.cc:492] T 6709e98d474f4acd9ceb70b0b7f74237 P 9117d1dafdbe4e6f804a6f7f742596b6: Bootstrap starting.
I20260811 19:12:04.358259 16884 tablet_bootstrap.cc:492] T 6709e98d474f4acd9ceb70b0b7f74237 P 06c375ae44c0466c8b91db22d0b0765c: Bootstrap starting.
I20260811 19:12:04.358464 16885 tablet_bootstrap.cc:492] T 6709e98d474f4acd9ceb70b0b7f74237 P 3e21de71b6e9439f80d484412bed0c8a: Bootstrap starting.
I20260811 19:12:04.362864 16887 tablet_bootstrap.cc:654] T 6709e98d474f4acd9ceb70b0b7f74237 P 9117d1dafdbe4e6f804a6f7f742596b6: Neither blocks nor log segments found. Creating new log.
I20260811 19:12:04.362851 16884 tablet_bootstrap.cc:654] T 6709e98d474f4acd9ceb70b0b7f74237 P 06c375ae44c0466c8b91db22d0b0765c: Neither blocks nor log segments found. Creating new log.
I20260811 19:12:04.363379 16885 tablet_bootstrap.cc:654] T 6709e98d474f4acd9ceb70b0b7f74237 P 3e21de71b6e9439f80d484412bed0c8a: Neither blocks nor log segments found. Creating new log.
I20260811 19:12:04.364727 16887 log.cc:824] T 6709e98d474f4acd9ceb70b0b7f74237 P 9117d1dafdbe4e6f804a6f7f742596b6: Log is configured to *not* fsync() on all Append() calls
I20260811 19:12:04.364729 16884 log.cc:824] T 6709e98d474f4acd9ceb70b0b7f74237 P 06c375ae44c0466c8b91db22d0b0765c: Log is configured to *not* fsync() on all Append() calls
I20260811 19:12:04.365041 16885 log.cc:824] T 6709e98d474f4acd9ceb70b0b7f74237 P 3e21de71b6e9439f80d484412bed0c8a: Log is configured to *not* fsync() on all Append() calls
I20260811 19:12:04.370448 16887 tablet_bootstrap.cc:492] T 6709e98d474f4acd9ceb70b0b7f74237 P 9117d1dafdbe4e6f804a6f7f742596b6: No bootstrap required, opened a new log
I20260811 19:12:04.370834 16884 tablet_bootstrap.cc:492] T 6709e98d474f4acd9ceb70b0b7f74237 P 06c375ae44c0466c8b91db22d0b0765c: No bootstrap required, opened a new log
I20260811 19:12:04.371012 16887 ts_tablet_manager.cc:1397] T 6709e98d474f4acd9ceb70b0b7f74237 P 9117d1dafdbe4e6f804a6f7f742596b6: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.006s
I20260811 19:12:04.371235 16884 ts_tablet_manager.cc:1397] T 6709e98d474f4acd9ceb70b0b7f74237 P 06c375ae44c0466c8b91db22d0b0765c: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.008s
I20260811 19:12:04.381743 16887 raft_consensus.cc:348] T 6709e98d474f4acd9ceb70b0b7f74237 P 9117d1dafdbe4e6f804a6f7f742596b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e21de71b6e9439f80d484412bed0c8a" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 38449 } } peers { permanent_uuid: "06c375ae44c0466c8b91db22d0b0765c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 32931 } } peers { permanent_uuid: "9117d1dafdbe4e6f804a6f7f742596b6" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 34125 } }
I20260811 19:12:04.382284 16885 tablet_bootstrap.cc:492] T 6709e98d474f4acd9ceb70b0b7f74237 P 3e21de71b6e9439f80d484412bed0c8a: No bootstrap required, opened a new log
I20260811 19:12:04.382355 16887 raft_consensus.cc:374] T 6709e98d474f4acd9ceb70b0b7f74237 P 9117d1dafdbe4e6f804a6f7f742596b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:12:04.382483 16887 raft_consensus.cc:729] T 6709e98d474f4acd9ceb70b0b7f74237 P 9117d1dafdbe4e6f804a6f7f742596b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9117d1dafdbe4e6f804a6f7f742596b6, State: Initialized, Role: FOLLOWER
I20260811 19:12:04.382665 16885 ts_tablet_manager.cc:1397] T 6709e98d474f4acd9ceb70b0b7f74237 P 3e21de71b6e9439f80d484412bed0c8a: Time spent bootstrapping tablet: real 0.025s user 0.009s sys 0.013s
I20260811 19:12:04.383271 16887 consensus_queue.cc:260] T 6709e98d474f4acd9ceb70b0b7f74237 P 9117d1dafdbe4e6f804a6f7f742596b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e21de71b6e9439f80d484412bed0c8a" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 38449 } } peers { permanent_uuid: "06c375ae44c0466c8b91db22d0b0765c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 32931 } } peers { permanent_uuid: "9117d1dafdbe4e6f804a6f7f742596b6" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 34125 } }
I20260811 19:12:04.384285 16884 raft_consensus.cc:348] T 6709e98d474f4acd9ceb70b0b7f74237 P 06c375ae44c0466c8b91db22d0b0765c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e21de71b6e9439f80d484412bed0c8a" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 38449 } } peers { permanent_uuid: "06c375ae44c0466c8b91db22d0b0765c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 32931 } } peers { permanent_uuid: "9117d1dafdbe4e6f804a6f7f742596b6" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 34125 } }
I20260811 19:12:04.385200 16884 raft_consensus.cc:374] T 6709e98d474f4acd9ceb70b0b7f74237 P 06c375ae44c0466c8b91db22d0b0765c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:12:04.385421 16822 heartbeater.cc:503] Master 127.3.61.190:33527 was elected leader, sending a full tablet report...
I20260811 19:12:04.385499 16884 raft_consensus.cc:729] T 6709e98d474f4acd9ceb70b0b7f74237 P 06c375ae44c0466c8b91db22d0b0765c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06c375ae44c0466c8b91db22d0b0765c, State: Initialized, Role: FOLLOWER
I20260811 19:12:04.386157 16887 ts_tablet_manager.cc:1428] T 6709e98d474f4acd9ceb70b0b7f74237 P 9117d1dafdbe4e6f804a6f7f742596b6: Time spent starting tablet: real 0.015s user 0.002s sys 0.013s
I20260811 19:12:04.387009 16884 consensus_queue.cc:260] T 6709e98d474f4acd9ceb70b0b7f74237 P 06c375ae44c0466c8b91db22d0b0765c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e21de71b6e9439f80d484412bed0c8a" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 38449 } } peers { permanent_uuid: "06c375ae44c0466c8b91db22d0b0765c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 32931 } } peers { permanent_uuid: "9117d1dafdbe4e6f804a6f7f742596b6" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 34125 } }
I20260811 19:12:04.390869 16580 heartbeater.cc:503] Master 127.3.61.190:33527 was elected leader, sending a full tablet report...
I20260811 19:12:04.391767 16884 ts_tablet_manager.cc:1428] T 6709e98d474f4acd9ceb70b0b7f74237 P 06c375ae44c0466c8b91db22d0b0765c: Time spent starting tablet: real 0.020s user 0.016s sys 0.001s
I20260811 19:12:04.397001 16885 raft_consensus.cc:348] T 6709e98d474f4acd9ceb70b0b7f74237 P 3e21de71b6e9439f80d484412bed0c8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e21de71b6e9439f80d484412bed0c8a" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 38449 } } peers { permanent_uuid: "06c375ae44c0466c8b91db22d0b0765c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 32931 } } peers { permanent_uuid: "9117d1dafdbe4e6f804a6f7f742596b6" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 34125 } }
I20260811 19:12:04.397574 16885 raft_consensus.cc:374] T 6709e98d474f4acd9ceb70b0b7f74237 P 3e21de71b6e9439f80d484412bed0c8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:12:04.397717 16885 raft_consensus.cc:729] T 6709e98d474f4acd9ceb70b0b7f74237 P 3e21de71b6e9439f80d484412bed0c8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e21de71b6e9439f80d484412bed0c8a, State: Initialized, Role: FOLLOWER
I20260811 19:12:04.398427 16885 consensus_queue.cc:260] T 6709e98d474f4acd9ceb70b0b7f74237 P 3e21de71b6e9439f80d484412bed0c8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e21de71b6e9439f80d484412bed0c8a" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 38449 } } peers { permanent_uuid: "06c375ae44c0466c8b91db22d0b0765c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 32931 } } peers { permanent_uuid: "9117d1dafdbe4e6f804a6f7f742596b6" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 34125 } }
I20260811 19:12:04.401011 16885 ts_tablet_manager.cc:1428] T 6709e98d474f4acd9ceb70b0b7f74237 P 3e21de71b6e9439f80d484412bed0c8a: Time spent starting tablet: real 0.018s user 0.013s sys 0.005s
W20260811 19:12:04.404023 16825 tablet.cc:2369] T 6709e98d474f4acd9ceb70b0b7f74237 P 9117d1dafdbe4e6f804a6f7f742596b6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:12:04.441807 16892 raft_consensus.cc:482] T 6709e98d474f4acd9ceb70b0b7f74237 P 9117d1dafdbe4e6f804a6f7f742596b6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:12:04.442221 16892 raft_consensus.cc:504] T 6709e98d474f4acd9ceb70b0b7f74237 P 9117d1dafdbe4e6f804a6f7f742596b6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e21de71b6e9439f80d484412bed0c8a" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 38449 } } peers { permanent_uuid: "06c375ae44c0466c8b91db22d0b0765c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 32931 } } peers { permanent_uuid: "9117d1dafdbe4e6f804a6f7f742596b6" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 34125 } }
I20260811 19:12:04.444689 16892 leader_election.cc:288] T 6709e98d474f4acd9ceb70b0b7f74237 P 9117d1dafdbe4e6f804a6f7f742596b6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e21de71b6e9439f80d484412bed0c8a (127.3.61.129:38449), 06c375ae44c0466c8b91db22d0b0765c (127.3.61.130:32931)
W20260811 19:12:04.446069 16588 tablet.cc:2369] T 6709e98d474f4acd9ceb70b0b7f74237 P 06c375ae44c0466c8b91db22d0b0765c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:12:04.453600 16880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:04.445180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:46003 (local address 127.3.61.130:32931)
0811 19:12:04.445390 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:04.445397 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:04.445699 (+ 302us) server_negotiation.cc:407] Connection header received
0811 19:12:04.445770 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:04.445776 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:04.445851 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:04.445917 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:12:04.446578 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:04.447700 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:04.450468 (+ 2768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:04.452359 (+ 1891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:04.452559 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:04.452639 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:04.453167 (+ 528us) server_negotiation.cc:299] Negotiation successful
0811 19:12:04.453315 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 19:12:04.453583 16882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:04.446236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:39801 (local address 127.3.61.129:38449)
0811 19:12:04.446490 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:04.446496 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:04.446516 (+ 20us) server_negotiation.cc:407] Connection header received
0811 19:12:04.446587 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:04.446592 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:04.446698 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:04.446796 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:12:04.447286 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:04.448223 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:04.450364 (+ 2141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:04.452254 (+ 1890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:04.452498 (+ 244us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:04.452596 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:04.453080 (+ 484us) server_negotiation.cc:299] Negotiation successful
0811 19:12:04.453270 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 19:12:04.453640 16835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:04.445034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.130:32931 (local address 127.3.61.131:46003)
0811 19:12:04.445374 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:04.445398 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:04.445561 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:04.446090 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:04.446095 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:04.446123 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:12:04.446402 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:04.446416 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:04.447846 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:04.447851 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:04.450267 (+ 2416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:04.450285 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:04.452582 (+ 2297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:04.452588 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:04.452979 (+ 391us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:04.453010 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 19:12:04.453094 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 19:12:04.453229 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":26}
I20260811 19:12:04.454094 16832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:04.445418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:38449 (local address 127.3.61.131:39801)
0811 19:12:04.445936 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:04.445960 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:04.446096 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:04.446841 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:04.446846 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:04.446873 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:12:04.447134 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:04.447146 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:04.448379 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:04.448383 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:04.450175 (+ 1792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:04.450190 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:04.452511 (+ 2321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:04.452523 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:04.452825 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:04.452866 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 19:12:04.453006 (+ 140us) client_negotiation.cc:240] Negotiation successful
0811 19:12:04.453187 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":403,"mutex_wait_us":38}
I20260811 19:12:04.455229 16278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6709e98d474f4acd9ceb70b0b7f74237" candidate_uuid: "9117d1dafdbe4e6f804a6f7f742596b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e21de71b6e9439f80d484412bed0c8a" is_pre_election: true
I20260811 19:12:04.455220 16506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6709e98d474f4acd9ceb70b0b7f74237" candidate_uuid: "9117d1dafdbe4e6f804a6f7f742596b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06c375ae44c0466c8b91db22d0b0765c" is_pre_election: true
I20260811 19:12:04.455845 16278 raft_consensus.cc:2454] T 6709e98d474f4acd9ceb70b0b7f74237 P 3e21de71b6e9439f80d484412bed0c8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9117d1dafdbe4e6f804a6f7f742596b6 in term 0.
I20260811 19:12:04.455932 16506 raft_consensus.cc:2454] T 6709e98d474f4acd9ceb70b0b7f74237 P 06c375ae44c0466c8b91db22d0b0765c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9117d1dafdbe4e6f804a6f7f742596b6 in term 0.
I20260811 19:12:04.456792 16631 leader_election.cc:302] T 6709e98d474f4acd9ceb70b0b7f74237 P 9117d1dafdbe4e6f804a6f7f742596b6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06c375ae44c0466c8b91db22d0b0765c, 9117d1dafdbe4e6f804a6f7f742596b6; no voters:
I20260811 19:12:04.457273 16892 raft_consensus.cc:2781] T 6709e98d474f4acd9ceb70b0b7f74237 P 9117d1dafdbe4e6f804a6f7f742596b6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:12:04.457491 16892 raft_consensus.cc:482] T 6709e98d474f4acd9ceb70b0b7f74237 P 9117d1dafdbe4e6f804a6f7f742596b6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:12:04.457619 16892 raft_consensus.cc:3037] T 6709e98d474f4acd9ceb70b0b7f74237 P 9117d1dafdbe4e6f804a6f7f742596b6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:12:04.460340 16892 raft_consensus.cc:504] T 6709e98d474f4acd9ceb70b0b7f74237 P 9117d1dafdbe4e6f804a6f7f742596b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e21de71b6e9439f80d484412bed0c8a" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 38449 } } peers { permanent_uuid: "06c375ae44c0466c8b91db22d0b0765c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 32931 } } peers { permanent_uuid: "9117d1dafdbe4e6f804a6f7f742596b6" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 34125 } }
I20260811 19:12:04.461066 16892 leader_election.cc:288] T 6709e98d474f4acd9ceb70b0b7f74237 P 9117d1dafdbe4e6f804a6f7f742596b6 [CANDIDATE]: Term 1 election: Requested vote from peers 3e21de71b6e9439f80d484412bed0c8a (127.3.61.129:38449), 06c375ae44c0466c8b91db22d0b0765c (127.3.61.130:32931)
I20260811 19:12:04.461730 16278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6709e98d474f4acd9ceb70b0b7f74237" candidate_uuid: "9117d1dafdbe4e6f804a6f7f742596b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e21de71b6e9439f80d484412bed0c8a"
I20260811 19:12:04.461947 16278 raft_consensus.cc:3037] T 6709e98d474f4acd9ceb70b0b7f74237 P 3e21de71b6e9439f80d484412bed0c8a [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:12:04.464316 16506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6709e98d474f4acd9ceb70b0b7f74237" candidate_uuid: "9117d1dafdbe4e6f804a6f7f742596b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06c375ae44c0466c8b91db22d0b0765c"
I20260811 19:12:04.464627 16278 raft_consensus.cc:2454] T 6709e98d474f4acd9ceb70b0b7f74237 P 3e21de71b6e9439f80d484412bed0c8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9117d1dafdbe4e6f804a6f7f742596b6 in term 1.
I20260811 19:12:04.464728 16506 raft_consensus.cc:3037] T 6709e98d474f4acd9ceb70b0b7f74237 P 06c375ae44c0466c8b91db22d0b0765c [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:12:04.465520 16634 leader_election.cc:302] T 6709e98d474f4acd9ceb70b0b7f74237 P 9117d1dafdbe4e6f804a6f7f742596b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e21de71b6e9439f80d484412bed0c8a, 9117d1dafdbe4e6f804a6f7f742596b6; no voters:
I20260811 19:12:04.466167 16892 raft_consensus.cc:2781] T 6709e98d474f4acd9ceb70b0b7f74237 P 9117d1dafdbe4e6f804a6f7f742596b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:12:04.467180 16892 raft_consensus.cc:686] T 6709e98d474f4acd9ceb70b0b7f74237 P 9117d1dafdbe4e6f804a6f7f742596b6 [term 1 LEADER]: Becoming Leader. State: Replica: 9117d1dafdbe4e6f804a6f7f742596b6, State: Running, Role: LEADER
I20260811 19:12:04.467911 16892 consensus_queue.cc:237] T 6709e98d474f4acd9ceb70b0b7f74237 P 9117d1dafdbe4e6f804a6f7f742596b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e21de71b6e9439f80d484412bed0c8a" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 38449 } } peers { permanent_uuid: "06c375ae44c0466c8b91db22d0b0765c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 32931 } } peers { permanent_uuid: "9117d1dafdbe4e6f804a6f7f742596b6" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 34125 } }
I20260811 19:12:04.468887 16506 raft_consensus.cc:2454] T 6709e98d474f4acd9ceb70b0b7f74237 P 06c375ae44c0466c8b91db22d0b0765c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9117d1dafdbe4e6f804a6f7f742596b6 in term 1.
I20260811 19:12:04.474822 14791 catalog_manager.cc:5495] T 6709e98d474f4acd9ceb70b0b7f74237 P 9117d1dafdbe4e6f804a6f7f742596b6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9117d1dafdbe4e6f804a6f7f742596b6 (127.3.61.131). New cstate: current_term: 1 leader_uuid: "9117d1dafdbe4e6f804a6f7f742596b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e21de71b6e9439f80d484412bed0c8a" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 38449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06c375ae44c0466c8b91db22d0b0765c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 32931 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9117d1dafdbe4e6f804a6f7f742596b6" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 34125 } health_report { overall_health: HEALTHY } } }
W20260811 19:12:04.483927 16361 tablet.cc:2369] T 6709e98d474f4acd9ceb70b0b7f74237 P 3e21de71b6e9439f80d484412bed0c8a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:12:04.527433 16883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:04.519990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36900 (local address 127.3.61.131:34125)
0811 19:12:04.520276 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:04.520282 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:04.520362 (+ 80us) server_negotiation.cc:407] Connection header received
0811 19:12:04.520778 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:04.520786 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:04.520925 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:04.521137 (+ 212us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:12:04.521686 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:04.522546 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:04.524119 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:04.524989 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:04.525131 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:04.526083 (+ 952us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:12:04.526673 (+ 590us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:12:04.526945 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:04.527085 (+ 140us) server_negotiation.cc:299] Negotiation successful
0811 19:12:04.527171 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 19:12:04.538136 16278 raft_consensus.cc:1261] T 6709e98d474f4acd9ceb70b0b7f74237 P 3e21de71b6e9439f80d484412bed0c8a [term 1 FOLLOWER]: Refusing update from remote peer 9117d1dafdbe4e6f804a6f7f742596b6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:12:04.538129 16506 raft_consensus.cc:1261] T 6709e98d474f4acd9ceb70b0b7f74237 P 06c375ae44c0466c8b91db22d0b0765c [term 1 FOLLOWER]: Refusing update from remote peer 9117d1dafdbe4e6f804a6f7f742596b6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:12:04.539314 16899 consensus_queue.cc:1035] T 6709e98d474f4acd9ceb70b0b7f74237 P 9117d1dafdbe4e6f804a6f7f742596b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e21de71b6e9439f80d484412bed0c8a" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 38449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:12:04.539894 16892 consensus_queue.cc:1035] T 6709e98d474f4acd9ceb70b0b7f74237 P 9117d1dafdbe4e6f804a6f7f742596b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06c375ae44c0466c8b91db22d0b0765c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 32931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:12:04.554785 16902 mvcc.cc:205] Tried to move back new op lower bound from 7317403748495081472 to 7317403748225310720. Current Snapshot: MvccSnapshot[applied={T|T < 7317403748495081472}]
I20260811 19:12:04.559607 16904 mvcc.cc:205] Tried to move back new op lower bound from 7317403748495081472 to 7317403748225310720. Current Snapshot: MvccSnapshot[applied={T|T < 7317403748495081472}]
I20260811 19:12:04.575650 14792 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:12:04.575981 14792 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:12:04.585227 14792 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d719d69f5e494d2eaecba22f5241c8b4: Sending DeleteTablet for 3 replicas of tablet 6709e98d474f4acd9ceb70b0b7f74237
I20260811 19:12:04.586593 16246 tablet_service.cc:1514] Processing DeleteTablet for tablet 6709e98d474f4acd9ceb70b0b7f74237 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:12:04 UTC) from {username='oryx', principal='oryx/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:38770
I20260811 19:12:04.586753 16473 tablet_service.cc:1514] Processing DeleteTablet for tablet 6709e98d474f4acd9ceb70b0b7f74237 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:12:04 UTC) from {username='oryx', principal='oryx/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:34444
I20260811 19:12:04.587915 16712 tablet_service.cc:1514] Processing DeleteTablet for tablet 6709e98d474f4acd9ceb70b0b7f74237 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:12:04 UTC) from {username='oryx', principal='oryx/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:36888
I20260811 19:12:04.589035 16920 tablet_replica.cc:330] T 6709e98d474f4acd9ceb70b0b7f74237 P 06c375ae44c0466c8b91db22d0b0765c: stopping tablet replica
I20260811 19:12:04.589578 16920 raft_consensus.cc:2229] T 6709e98d474f4acd9ceb70b0b7f74237 P 06c375ae44c0466c8b91db22d0b0765c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:12:04.590341 16920 raft_consensus.cc:2258] T 6709e98d474f4acd9ceb70b0b7f74237 P 06c375ae44c0466c8b91db22d0b0765c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:12:04.593847 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 14736
I20260811 19:12:04.596258 14841 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:12:04.597198 16920 ts_tablet_manager.cc:1905] T 6709e98d474f4acd9ceb70b0b7f74237 P 06c375ae44c0466c8b91db22d0b0765c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:12:04.606855 16921 tablet_replica.cc:330] T 6709e98d474f4acd9ceb70b0b7f74237 P 9117d1dafdbe4e6f804a6f7f742596b6: stopping tablet replica
I20260811 19:12:04.607853 16921 raft_consensus.cc:2229] T 6709e98d474f4acd9ceb70b0b7f74237 P 9117d1dafdbe4e6f804a6f7f742596b6 [term 1 LEADER]: Raft consensus shutting down.
I20260811 19:12:04.609030 16921 raft_consensus.cc:2258] T 6709e98d474f4acd9ceb70b0b7f74237 P 9117d1dafdbe4e6f804a6f7f742596b6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:12:04.610535 16918 tablet_replica.cc:330] T 6709e98d474f4acd9ceb70b0b7f74237 P 3e21de71b6e9439f80d484412bed0c8a: stopping tablet replica
I20260811 19:12:04.611218 16918 raft_consensus.cc:2229] T 6709e98d474f4acd9ceb70b0b7f74237 P 3e21de71b6e9439f80d484412bed0c8a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:12:04.612040 16918 raft_consensus.cc:2258] T 6709e98d474f4acd9ceb70b0b7f74237 P 3e21de71b6e9439f80d484412bed0c8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:12:04.612881 16921 ts_tablet_manager.cc:1905] T 6709e98d474f4acd9ceb70b0b7f74237 P 9117d1dafdbe4e6f804a6f7f742596b6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:12:04.615128 16918 ts_tablet_manager.cc:1905] T 6709e98d474f4acd9ceb70b0b7f74237 P 3e21de71b6e9439f80d484412bed0c8a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:12:04.617430 16920 ts_tablet_manager.cc:1918] T 6709e98d474f4acd9ceb70b0b7f74237 P 06c375ae44c0466c8b91db22d0b0765c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:12:04.617723 16920 log.cc:1196] T 6709e98d474f4acd9ceb70b0b7f74237 P 06c375ae44c0466c8b91db22d0b0765c: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-1/wal/wals/6709e98d474f4acd9ceb70b0b7f74237
I20260811 19:12:04.618475 16920 ts_tablet_manager.cc:1939] T 6709e98d474f4acd9ceb70b0b7f74237 P 06c375ae44c0466c8b91db22d0b0765c: Deleting consensus metadata
I20260811 19:12:04.624665 16921 ts_tablet_manager.cc:1918] T 6709e98d474f4acd9ceb70b0b7f74237 P 9117d1dafdbe4e6f804a6f7f742596b6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:12:04.625073 16921 log.cc:1196] T 6709e98d474f4acd9ceb70b0b7f74237 P 9117d1dafdbe4e6f804a6f7f742596b6: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-2/wal/wals/6709e98d474f4acd9ceb70b0b7f74237
I20260811 19:12:04.626046 16921 ts_tablet_manager.cc:1939] T 6709e98d474f4acd9ceb70b0b7f74237 P 9117d1dafdbe4e6f804a6f7f742596b6: Deleting consensus metadata
I20260811 19:12:04.630280 16918 ts_tablet_manager.cc:1918] T 6709e98d474f4acd9ceb70b0b7f74237 P 3e21de71b6e9439f80d484412bed0c8a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:12:04.630541 16918 log.cc:1196] T 6709e98d474f4acd9ceb70b0b7f74237 P 3e21de71b6e9439f80d484412bed0c8a: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-0/wal/wals/6709e98d474f4acd9ceb70b0b7f74237
I20260811 19:12:04.631302 16918 ts_tablet_manager.cc:1939] T 6709e98d474f4acd9ceb70b0b7f74237 P 3e21de71b6e9439f80d484412bed0c8a: Deleting consensus metadata
I20260811 19:12:04.635928 16848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:04.589867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47356 (local address 127.3.61.189:43587)
0811 19:12:04.590181 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:04.590188 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:04.592942 (+ 2754us) server_negotiation.cc:407] Connection header received
0811 19:12:04.594440 (+ 1498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:04.594446 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:04.594553 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:04.594638 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:04.612860 (+ 18222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:04.614060 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:04.620291 (+ 6231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:04.621144 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:04.621267 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:04.626315 (+ 5048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:04.626324 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:04.626330 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:04.626381 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:04.631659 (+ 5278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:04.632781 (+ 1122us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:04.632787 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:04.632791 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:04.632983 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:04.633550 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:04.633556 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:04.633559 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:04.634028 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:04.634292 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:04.635244 (+ 952us) server_negotiation.cc:299] Negotiation successful
0811 19:12:04.635536 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 19:12:04.658928 16836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:04.593380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34008 (local address 127.3.61.188:46509)
0811 19:12:04.593563 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:04.593569 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:04.638019 (+ 44450us) server_negotiation.cc:407] Connection header received
0811 19:12:04.638273 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:04.638279 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:04.638444 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:04.638520 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:04.641033 (+ 2513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:04.642270 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:04.645247 (+ 2977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:04.646087 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:04.646216 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:04.650307 (+ 4091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:04.650314 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:04.650320 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:04.650366 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:04.655029 (+ 4663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:04.656117 (+ 1088us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:04.656124 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:04.656128 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:04.656336 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:04.657067 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:04.657073 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:04.657077 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:04.657550 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:04.657860 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:04.658243 (+ 383us) server_negotiation.cc:299] Negotiation successful
0811 19:12:04.658493 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 19:12:04.759394 14765 catalog_manager.cc:4841] TS 06c375ae44c0466c8b91db22d0b0765c (127.3.61.130:32931): tablet 6709e98d474f4acd9ceb70b0b7f74237 (table test-table [id=802b55ec40874b53b0b1f03ae854a4bd]) successfully deleted
I20260811 19:12:04.759562 14767 catalog_manager.cc:4841] TS 9117d1dafdbe4e6f804a6f7f742596b6 (127.3.61.131:34125): tablet 6709e98d474f4acd9ceb70b0b7f74237 (table test-table [id=802b55ec40874b53b0b1f03ae854a4bd]) successfully deleted
I20260811 19:12:04.759994 14768 catalog_manager.cc:4841] TS 3e21de71b6e9439f80d484412bed0c8a (127.3.61.129:38449): tablet 6709e98d474f4acd9ceb70b0b7f74237 (table test-table [id=802b55ec40874b53b0b1f03ae854a4bd]) successfully deleted
I20260811 19:12:04.760183 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 14736
I20260811 19:12:04.826772 16848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:04.804801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47360 (local address 127.3.61.189:43587)
0811 19:12:04.805033 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:04.805049 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:04.806765 (+ 1716us) server_negotiation.cc:407] Connection header received
0811 19:12:04.806966 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:04.806987 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:04.807100 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:04.807248 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:04.810674 (+ 3426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:04.811609 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:04.813549 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:04.814344 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:04.814540 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:04.818761 (+ 4221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:04.818768 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:04.818772 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:04.818814 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:04.823025 (+ 4211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:04.824083 (+ 1058us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:04.824089 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:04.824093 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:04.824272 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:04.824765 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:04.824771 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:04.824774 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:04.825249 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:04.825531 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:04.826054 (+ 523us) server_negotiation.cc:299] Negotiation successful
0811 19:12:04.826382 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 19:12:04.828984 16836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:04.806351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34024 (local address 127.3.61.188:46509)
0811 19:12:04.806555 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:04.806561 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:04.808081 (+ 1520us) server_negotiation.cc:407] Connection header received
0811 19:12:04.808259 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:04.808264 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:04.808339 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:04.808437 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:04.810920 (+ 2483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:04.812126 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:04.813970 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:04.814763 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:04.814931 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:04.819744 (+ 4813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:04.819753 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:04.819758 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:04.819805 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:04.823894 (+ 4089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:04.825335 (+ 1441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:04.825351 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:04.825355 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:04.825558 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:04.827071 (+ 1513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:04.827077 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:04.827081 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:04.827545 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:04.827846 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:04.828373 (+ 527us) server_negotiation.cc:299] Negotiation successful
0811 19:12:04.828644 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 19:12:05.562026 16932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:05.561055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33527 (local address 127.3.61.131:59637)
0811 19:12:05.561598 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:05.561762 (+ 164us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:33527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":304,"threads_started":1}
W20260811 19:12:05.562463 16822 heartbeater.cc:644] Failed to heartbeat to 127.3.61.190:33527 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.3.61.190:33527: connect: Connection refused (error 111)
I20260811 19:12:05.566800 16933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:05.565954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33527 (local address 127.3.61.130:48327)
0811 19:12:05.566440 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:05.566602 (+ 162us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:33527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":245,"threads_started":1}
W20260811 19:12:05.567452 16580 heartbeater.cc:644] Failed to heartbeat to 127.3.61.190:33527 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.3.61.190:33527: connect: Connection refused (error 111)
I20260811 19:12:05.568455 16934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:05.567584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33527 (local address 127.3.61.129:41009)
0811 19:12:05.568100 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:05.568242 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:33527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":284,"threads_started":1}
W20260811 19:12:05.568858 16358 heartbeater.cc:644] Failed to heartbeat to 127.3.61.190:33527 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.3.61.190:33527: connect: Connection refused (error 111)
I20260811 19:12:06.084233 16935 raft_consensus.cc:482] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d719d69f5e494d2eaecba22f5241c8b4)
I20260811 19:12:06.084589 16935 raft_consensus.cc:504] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: false } } peers { permanent_uuid: "006be47d760e41be8683477553cb2bf3" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 46509 } attrs { promote: false } }
I20260811 19:12:06.084966 16936 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d719d69f5e494d2eaecba22f5241c8b4)
I20260811 19:12:06.085238 16936 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: false } } peers { permanent_uuid: "006be47d760e41be8683477553cb2bf3" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 46509 } attrs { promote: false } }
I20260811 19:12:06.086364 16935 leader_election.cc:288] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d719d69f5e494d2eaecba22f5241c8b4 (127.3.61.190:33527), 0c62dbd92db642fa9b56acc74117c8c0 (127.3.61.189:43587)
I20260811 19:12:06.086588 16936 leader_election.cc:288] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d719d69f5e494d2eaecba22f5241c8b4 (127.3.61.190:33527), 006be47d760e41be8683477553cb2bf3 (127.3.61.188:46509)
I20260811 19:12:06.099623 16944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:06.086745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33527 (local address 127.0.0.1:48834)
0811 19:12:06.098875 (+ 12130us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:06.099225 (+ 350us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:33527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11918,"threads_started":1}
I20260811 19:12:06.100337 16945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:06.086997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33527 (local address 127.0.0.1:48838)
0811 19:12:06.099725 (+ 12728us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:06.100083 (+ 358us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:33527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12630,"thread_start_us":12491,"threads_started":1}
W20260811 19:12:06.100270 16044 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.61.190:33527: connect: Connection refused (error 111)
W20260811 19:12:06.102471 15742 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.61.190:33527: connect: Connection refused (error 111)
I20260811 19:12:06.103756 16944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:06.102993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33527 (local address 127.0.0.1:48844)
0811 19:12:06.103337 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:06.103482 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:33527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168}
W20260811 19:12:06.104265 16044 leader_election.cc:334] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d719d69f5e494d2eaecba22f5241c8b4 (127.3.61.190:33527): Network error: Client connection negotiation failed: client connection to 127.3.61.190:33527: connect: Connection refused (error 111)
I20260811 19:12:06.107252 16955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:06.105714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33527 (local address 127.0.0.1:48858)
0811 19:12:06.106786 (+ 1072us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:06.106969 (+ 183us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:33527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":847,"thread_start_us":587,"threads_started":1}
W20260811 19:12:06.107553 15742 leader_election.cc:334] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d719d69f5e494d2eaecba22f5241c8b4 (127.3.61.190:33527): Network error: Client connection negotiation failed: client connection to 127.3.61.190:33527: connect: Connection refused (error 111)
I20260811 19:12:06.108588 16937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:06.086806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.189:43587 (local address 127.0.0.1:47372)
0811 19:12:06.099218 (+ 12412us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:06.099241 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:06.099453 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:06.099963 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:06.099968 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:06.099996 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:12:06.100467 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:06.100481 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:06.102869 (+ 2388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:06.102875 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:06.105353 (+ 2478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:06.105369 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:06.107672 (+ 2303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:06.107680 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:06.107905 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:06.107960 (+ 55us) client_negotiation.cc:769] Sending connection context
0811 19:12:06.108081 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 19:12:06.108249 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12257,"mutex_wait_us":1,"thread_start_us":11644,"threads_started":1}
I20260811 19:12:06.109213 16942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:06.086809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47372 (local address 127.3.61.189:43587)
0811 19:12:06.099226 (+ 12417us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:06.099233 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:06.099592 (+ 359us) server_negotiation.cc:407] Connection header received
0811 19:12:06.099694 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:06.099700 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:06.099777 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:06.100636 (+ 859us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:12:06.101167 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:06.102640 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:06.105531 (+ 2891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:06.107350 (+ 1819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:06.107615 (+ 265us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:06.107727 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:06.108825 (+ 1098us) server_negotiation.cc:299] Negotiation successful
0811 19:12:06.108983 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":12294,"thread_start_us":12421,"threads_started":1}
I20260811 19:12:06.109951 15773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "006be47d760e41be8683477553cb2bf3" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "0c62dbd92db642fa9b56acc74117c8c0" is_pre_election: true
I20260811 19:12:06.110213 16945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:06.101361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:46509 (local address 127.0.0.1:34036)
0811 19:12:06.101647 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:06.101695 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:06.101905 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:06.102640 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:06.102646 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:06.102705 (+ 59us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:12:06.103209 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:06.103223 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:06.105237 (+ 2014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:06.105248 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:06.107990 (+ 2742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:06.108006 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:06.109548 (+ 1542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:06.109554 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:06.109779 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:06.109806 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 19:12:06.109885 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 19:12:06.110009 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73,"mutex_wait_us":69}
I20260811 19:12:06.110368 15773 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 006be47d760e41be8683477553cb2bf3 in term 1.
I20260811 19:12:06.110582 16946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:06.100236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34036 (local address 127.3.61.188:46509)
0811 19:12:06.101440 (+ 1204us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:06.101447 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:06.102063 (+ 616us) server_negotiation.cc:407] Connection header received
0811 19:12:06.102158 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:06.102163 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:06.102255 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:06.102367 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:12:06.103463 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:06.105052 (+ 1589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:06.108168 (+ 3116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:06.109349 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:06.109480 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:06.109563 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:06.109968 (+ 405us) server_negotiation.cc:299] Negotiation successful
0811 19:12:06.110129 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1033,"thread_start_us":702,"threads_started":1}
I20260811 19:12:06.111192 16040 leader_election.cc:302] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 006be47d760e41be8683477553cb2bf3, 0c62dbd92db642fa9b56acc74117c8c0; no voters: d719d69f5e494d2eaecba22f5241c8b4
I20260811 19:12:06.111235 16082 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0c62dbd92db642fa9b56acc74117c8c0" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "006be47d760e41be8683477553cb2bf3" is_pre_election: true
I20260811 19:12:06.111591 16082 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c62dbd92db642fa9b56acc74117c8c0 in term 1.
I20260811 19:12:06.111891 16935 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 19:12:06.112035 16935 raft_consensus.cc:482] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d719d69f5e494d2eaecba22f5241c8b4)
I20260811 19:12:06.112133 16935 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [term 1 FOLLOWER]: Advancing to term 2
I20260811 19:12:06.112236 15742 leader_election.cc:302] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 006be47d760e41be8683477553cb2bf3, 0c62dbd92db642fa9b56acc74117c8c0; no voters: d719d69f5e494d2eaecba22f5241c8b4
I20260811 19:12:06.112594 16936 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 19:12:06.112740 16936 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d719d69f5e494d2eaecba22f5241c8b4)
I20260811 19:12:06.112850 16936 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0 [term 1 FOLLOWER]: Advancing to term 2
I20260811 19:12:06.115388 16935 raft_consensus.cc:504] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: false } } peers { permanent_uuid: "006be47d760e41be8683477553cb2bf3" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 46509 } attrs { promote: false } }
I20260811 19:12:06.116277 16935 leader_election.cc:288] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [CANDIDATE]: Term 2 election: Requested vote from peers d719d69f5e494d2eaecba22f5241c8b4 (127.3.61.190:33527), 0c62dbd92db642fa9b56acc74117c8c0 (127.3.61.189:43587)
I20260811 19:12:06.117282 15773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "006be47d760e41be8683477553cb2bf3" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "0c62dbd92db642fa9b56acc74117c8c0"
I20260811 19:12:06.117524 16937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:06.116690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33527 (local address 127.0.0.1:48866)
0811 19:12:06.117157 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:06.117301 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:33527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":285}
I20260811 19:12:06.117693 16936 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: false } } peers { permanent_uuid: "006be47d760e41be8683477553cb2bf3" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 46509 } attrs { promote: false } }
I20260811 19:12:06.118238 15773 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 006be47d760e41be8683477553cb2bf3 in current term 2: Already voted for candidate 0c62dbd92db642fa9b56acc74117c8c0 in this term.
I20260811 19:12:06.118559 16936 leader_election.cc:288] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0 [CANDIDATE]: Term 2 election: Requested vote from peers d719d69f5e494d2eaecba22f5241c8b4 (127.3.61.190:33527), 006be47d760e41be8683477553cb2bf3 (127.3.61.188:46509)
I20260811 19:12:06.118860 16937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:06.118377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33527 (local address 127.0.0.1:48872)
0811 19:12:06.118535 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:06.118669 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:33527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 19:12:06.119166 16044 leader_election.cc:334] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer d719d69f5e494d2eaecba22f5241c8b4 (127.3.61.190:33527): Network error: Client connection negotiation failed: client connection to 127.3.61.190:33527: connect: Connection refused (error 111)
I20260811 19:12:06.119342 16044 leader_election.cc:302] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 006be47d760e41be8683477553cb2bf3; no voters: 0c62dbd92db642fa9b56acc74117c8c0, d719d69f5e494d2eaecba22f5241c8b4
I20260811 19:12:06.119498 16945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:06.118925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33527 (local address 127.0.0.1:48884)
0811 19:12:06.119177 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:06.119294 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:33527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260811 19:12:06.119609 16082 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0c62dbd92db642fa9b56acc74117c8c0" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "006be47d760e41be8683477553cb2bf3"
I20260811 19:12:06.119953 16082 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 0c62dbd92db642fa9b56acc74117c8c0 in current term 2: Already voted for candidate 006be47d760e41be8683477553cb2bf3 in this term.
I20260811 19:12:06.120163 16935 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260811 19:12:06.121707 16945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:06.121215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33527 (local address 127.0.0.1:48886)
0811 19:12:06.121408 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:06.121517 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:33527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260811 19:12:06.121937 15742 leader_election.cc:334] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer d719d69f5e494d2eaecba22f5241c8b4 (127.3.61.190:33527): Network error: Client connection negotiation failed: client connection to 127.3.61.190:33527: connect: Connection refused (error 111)
I20260811 19:12:06.122102 15742 leader_election.cc:302] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0c62dbd92db642fa9b56acc74117c8c0; no voters: 006be47d760e41be8683477553cb2bf3, d719d69f5e494d2eaecba22f5241c8b4
I20260811 19:12:06.122543 16936 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260811 19:12:06.564358 16956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:06.563362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33527 (local address 127.3.61.131:50325)
0811 19:12:06.563929 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:06.564056 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:33527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":304,"threads_started":1}
I20260811 19:12:06.569296 16958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:06.568470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33527 (local address 127.3.61.130:45365)
0811 19:12:06.568966 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:06.569061 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:33527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":251,"threads_started":1}
I20260811 19:12:06.570236 16960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:06.569507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33527 (local address 127.3.61.129:34607)
0811 19:12:06.569938 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:06.570016 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:33527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":243,"threads_started":1}
I20260811 19:12:07.566926 16961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:07.565538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33527 (local address 127.3.61.131:53397)
0811 19:12:07.566383 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:07.566616 (+ 233us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:33527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":690,"thread_start_us":457,"threads_started":1}
I20260811 19:12:07.571185 16963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:07.570327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33527 (local address 127.3.61.130:39971)
0811 19:12:07.570854 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:07.570960 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:33527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":295,"threads_started":1}
I20260811 19:12:07.572114 16964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:07.571308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33527 (local address 127.3.61.129:40479)
0811 19:12:07.571810 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:07.571912 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:33527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":258,"threads_started":1}
I20260811 19:12:08.078228 16966 raft_consensus.cc:482] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:12:08.078562 16966 raft_consensus.cc:504] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: false } } peers { permanent_uuid: "006be47d760e41be8683477553cb2bf3" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 46509 } attrs { promote: false } }
I20260811 19:12:08.079420 16966 leader_election.cc:288] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d719d69f5e494d2eaecba22f5241c8b4 (127.3.61.190:33527), 0c62dbd92db642fa9b56acc74117c8c0 (127.3.61.189:43587)
I20260811 19:12:08.080296 15773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "006be47d760e41be8683477553cb2bf3" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "0c62dbd92db642fa9b56acc74117c8c0" is_pre_election: true
I20260811 19:12:08.080633 15773 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 006be47d760e41be8683477553cb2bf3 in term 2.
I20260811 19:12:08.080847 16968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:08.079702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33527 (local address 127.0.0.1:49170)
0811 19:12:08.080300 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:08.080526 (+ 226us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:33527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":287,"threads_started":1}
I20260811 19:12:08.081214 16040 leader_election.cc:302] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 006be47d760e41be8683477553cb2bf3, 0c62dbd92db642fa9b56acc74117c8c0; no voters:
I20260811 19:12:08.081529 16966 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260811 19:12:08.081686 16966 raft_consensus.cc:482] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:12:08.081799 16966 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [term 2 FOLLOWER]: Advancing to term 3
I20260811 19:12:08.082784 16968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:08.082151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33527 (local address 127.0.0.1:49182)
0811 19:12:08.082338 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:08.082534 (+ 196us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:33527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 19:12:08.083206 16044 leader_election.cc:334] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d719d69f5e494d2eaecba22f5241c8b4 (127.3.61.190:33527): Network error: Client connection negotiation failed: client connection to 127.3.61.190:33527: connect: Connection refused (error 111)
I20260811 19:12:08.085013 16966 raft_consensus.cc:504] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: false } } peers { permanent_uuid: "006be47d760e41be8683477553cb2bf3" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 46509 } attrs { promote: false } }
I20260811 19:12:08.085762 16966 leader_election.cc:288] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [CANDIDATE]: Term 3 election: Requested vote from peers d719d69f5e494d2eaecba22f5241c8b4 (127.3.61.190:33527), 0c62dbd92db642fa9b56acc74117c8c0 (127.3.61.189:43587)
I20260811 19:12:08.086645 16968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:08.086039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33527 (local address 127.0.0.1:49186)
0811 19:12:08.086233 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:08.086340 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:33527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 19:12:08.086714 15773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "006be47d760e41be8683477553cb2bf3" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "0c62dbd92db642fa9b56acc74117c8c0"
I20260811 19:12:08.088099 16968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:08.087623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33527 (local address 127.0.0.1:49192)
0811 19:12:08.087802 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:08.087921 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:33527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260811 19:12:08.088181 15773 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0 [term 2 FOLLOWER]: Advancing to term 3
W20260811 19:12:08.088266 16044 leader_election.cc:334] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer d719d69f5e494d2eaecba22f5241c8b4 (127.3.61.190:33527): Network error: Client connection negotiation failed: client connection to 127.3.61.190:33527: connect: Connection refused (error 111)
I20260811 19:12:08.090478 15773 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 006be47d760e41be8683477553cb2bf3 in term 3.
I20260811 19:12:08.091043 16040 leader_election.cc:302] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 006be47d760e41be8683477553cb2bf3, 0c62dbd92db642fa9b56acc74117c8c0; no voters: d719d69f5e494d2eaecba22f5241c8b4
I20260811 19:12:08.091500 16966 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 19:12:08.092015 16966 raft_consensus.cc:686] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [term 3 LEADER]: Becoming Leader. State: Replica: 006be47d760e41be8683477553cb2bf3, State: Running, Role: LEADER
I20260811 19:12:08.092422 16966 consensus_queue.cc:237] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: false } } peers { permanent_uuid: "006be47d760e41be8683477553cb2bf3" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 46509 } attrs { promote: false } }
I20260811 19:12:08.094259 16971 sys_catalog.cc:455] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 006be47d760e41be8683477553cb2bf3. Latest consensus state: current_term: 3 leader_uuid: "006be47d760e41be8683477553cb2bf3" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: false } } peers { permanent_uuid: "006be47d760e41be8683477553cb2bf3" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 46509 } attrs { promote: false } } }
I20260811 19:12:08.094587 16971 sys_catalog.cc:458] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [sys.catalog]: This master's current role is: LEADER
I20260811 19:12:08.095235 16974 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:12:08.099663 16974 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:12:08.100672 16974 catalog_manager.cc:1259] Loaded cluster ID: e24c3dbff427436d95a3345db6017754
I20260811 19:12:08.100764 16974 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:12:08.102684 16974 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:12:08.104080 16974 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3: Loaded TSK: 0
I20260811 19:12:08.105353 16974 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:12:08.142397 16064 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34024:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 19:12:08.151144 16064 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:12:08.156690 16968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:08.156068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33527 (local address 127.0.0.1:49220)
0811 19:12:08.156274 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:08.156356 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:33527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260811 19:12:08.156836 15773 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0 [term 3 FOLLOWER]: Refusing update from remote peer 006be47d760e41be8683477553cb2bf3: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260811 19:12:08.157819 16968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:08.157401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33527 (local address 127.0.0.1:49232)
0811 19:12:08.157585 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:08.157693 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:33527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 19:12:08.157641 16971 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260811 19:12:08.158213 16044 consensus_peers.cc:487] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 -> Peer d719d69f5e494d2eaecba22f5241c8b4 (127.3.61.190:33527): Couldn't send request to peer d719d69f5e494d2eaecba22f5241c8b4. Status: Network error: Client connection negotiation failed: client connection to 127.3.61.190:33527: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 19:12:08.162001 16977 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 006be47d760e41be8683477553cb2bf3. Latest consensus state: current_term: 3 leader_uuid: "006be47d760e41be8683477553cb2bf3" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: false } } peers { permanent_uuid: "006be47d760e41be8683477553cb2bf3" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 46509 } attrs { promote: false } } }
I20260811 19:12:08.162463 16977 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:12:08.164754 16966 sys_catalog.cc:455] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "006be47d760e41be8683477553cb2bf3" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: false } } peers { permanent_uuid: "006be47d760e41be8683477553cb2bf3" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 46509 } attrs { promote: false } } }
I20260811 19:12:08.165262 16966 sys_catalog.cc:458] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3 [sys.catalog]: This master's current role is: LEADER
I20260811 19:12:08.167999 16977 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "006be47d760e41be8683477553cb2bf3" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d719d69f5e494d2eaecba22f5241c8b4" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 33527 } } peers { permanent_uuid: "0c62dbd92db642fa9b56acc74117c8c0" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43587 } attrs { promote: false } } peers { permanent_uuid: "006be47d760e41be8683477553cb2bf3" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 46509 } attrs { promote: false } } }
I20260811 19:12:08.168365 16977 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c62dbd92db642fa9b56acc74117c8c0 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:12:08.168978 16978 mvcc.cc:205] Tried to move back new op lower bound from 7317403763322208256 to 7317403763070300160. Current Snapshot: MvccSnapshot[applied={T|T < 7317403763322208256 or (T in {7317403763322208256})}]
I20260811 19:12:08.191543 16990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:08.176560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38772 (local address 127.3.61.129:38449)
0811 19:12:08.177061 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:08.177067 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:08.182455 (+ 5388us) server_negotiation.cc:407] Connection header received
0811 19:12:08.182700 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:08.182706 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:08.182797 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:08.182923 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:12:08.183847 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:08.185156 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:08.188598 (+ 3442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:08.190119 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:08.190239 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:08.190360 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:08.191018 (+ 658us) server_negotiation.cc:299] Negotiation successful
0811 19:12:08.191185 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":263,"threads_started":1}
I20260811 19:12:08.192355 16985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:08.176116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:38449 (local address 127.0.0.1:38772)
0811 19:12:08.182246 (+ 6130us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:08.182278 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:08.182519 (+ 241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:08.183135 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:08.183143 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:08.183178 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:12:08.183629 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:08.183642 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:08.185357 (+ 1715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:08.185362 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:08.188357 (+ 2995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:08.188376 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:08.190571 (+ 2195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:08.190579 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:08.190793 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:08.190835 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 19:12:08.191845 (+ 1010us) client_negotiation.cc:240] Negotiation successful
0811 19:12:08.191990 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5959,"thread_start_us":5707,"threads_started":1}
I20260811 19:12:08.194792 16246 tablet_service.cc:1467] Processing CreateTablet for tablet 1cb5db17f1e943b9984b5e458c6ecf76 (DEFAULT_TABLE table=test-table [id=5a1950f00bbd45f99683baf6f55eb4f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:12:08.195333 16246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1cb5db17f1e943b9984b5e458c6ecf76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:12:08.199261 16992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:08.178523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34446 (local address 127.3.61.130:32931)
0811 19:12:08.191729 (+ 13206us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:08.191735 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:08.191755 (+ 20us) server_negotiation.cc:407] Connection header received
0811 19:12:08.191828 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:08.191834 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:08.191909 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:08.192007 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:12:08.192818 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:08.193897 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:08.196408 (+ 2511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:08.197610 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:08.197758 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:08.197872 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:08.198753 (+ 881us) server_negotiation.cc:299] Negotiation successful
0811 19:12:08.198917 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13067,"thread_start_us":13454,"threads_started":1}
I20260811 19:12:08.199419 16991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:08.177921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.130:32931 (local address 127.0.0.1:34446)
0811 19:12:08.191123 (+ 13202us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:08.191145 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:08.191280 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:08.192193 (+ 913us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:08.192198 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:08.192227 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:12:08.192635 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:08.192646 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:08.194145 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:08.194151 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:08.196180 (+ 2029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:08.196199 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:08.198030 (+ 1831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:08.198036 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:08.198451 (+ 415us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:08.198508 (+ 57us) client_negotiation.cc:769] Sending connection context
0811 19:12:08.198709 (+ 201us) client_negotiation.cc:240] Negotiation successful
0811 19:12:08.198913 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13055,"thread_start_us":12901,"threads_started":1}
I20260811 19:12:08.200717 16473 tablet_service.cc:1467] Processing CreateTablet for tablet 1cb5db17f1e943b9984b5e458c6ecf76 (DEFAULT_TABLE table=test-table [id=5a1950f00bbd45f99683baf6f55eb4f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:12:08.201411 16473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1cb5db17f1e943b9984b5e458c6ecf76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:12:08.202739 16968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:08.175755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.131:34125 (local address 127.0.0.1:36916)
0811 19:12:08.176149 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:08.176170 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:08.176304 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:08.185612 (+ 9308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:08.185617 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:08.185680 (+ 63us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:12:08.186122 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:08.186135 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:08.193058 (+ 6923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:08.193064 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:08.197004 (+ 3940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:08.197019 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:08.201039 (+ 4020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:08.201046 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:08.201550 (+ 504us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:08.201795 (+ 245us) client_negotiation.cc:769] Sending connection context
0811 19:12:08.201984 (+ 189us) client_negotiation.cc:240] Negotiation successful
0811 19:12:08.202308 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274}
I20260811 19:12:08.203285 16994 tablet_bootstrap.cc:492] T 1cb5db17f1e943b9984b5e458c6ecf76 P 3e21de71b6e9439f80d484412bed0c8a: Bootstrap starting.
I20260811 19:12:08.203677 16989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:08.176193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36916 (local address 127.3.61.131:34125)
0811 19:12:08.184046 (+ 7853us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:08.184053 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:08.184082 (+ 29us) server_negotiation.cc:407] Connection header received
0811 19:12:08.184158 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:08.184164 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:08.184241 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:08.186292 (+ 2051us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:12:08.186490 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:08.187540 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:08.197256 (+ 9716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:08.200725 (+ 3469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:08.201062 (+ 337us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:08.201289 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:08.203123 (+ 1834us) server_negotiation.cc:299] Negotiation successful
0811 19:12:08.203323 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7694,"thread_start_us":11871,"threads_started":1}
I20260811 19:12:08.204653 16712 tablet_service.cc:1467] Processing CreateTablet for tablet 1cb5db17f1e943b9984b5e458c6ecf76 (DEFAULT_TABLE table=test-table [id=5a1950f00bbd45f99683baf6f55eb4f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:12:08.205185 16712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1cb5db17f1e943b9984b5e458c6ecf76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:12:08.205448 16994 tablet_bootstrap.cc:654] T 1cb5db17f1e943b9984b5e458c6ecf76 P 3e21de71b6e9439f80d484412bed0c8a: Neither blocks nor log segments found. Creating new log.
I20260811 19:12:08.208389 16994 tablet_bootstrap.cc:492] T 1cb5db17f1e943b9984b5e458c6ecf76 P 3e21de71b6e9439f80d484412bed0c8a: No bootstrap required, opened a new log
I20260811 19:12:08.208556 16994 ts_tablet_manager.cc:1397] T 1cb5db17f1e943b9984b5e458c6ecf76 P 3e21de71b6e9439f80d484412bed0c8a: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 19:12:08.209395 16994 raft_consensus.cc:348] T 1cb5db17f1e943b9984b5e458c6ecf76 P 3e21de71b6e9439f80d484412bed0c8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9117d1dafdbe4e6f804a6f7f742596b6" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 34125 } } peers { permanent_uuid: "3e21de71b6e9439f80d484412bed0c8a" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 38449 } } peers { permanent_uuid: "06c375ae44c0466c8b91db22d0b0765c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 32931 } }
I20260811 19:12:08.209625 16994 raft_consensus.cc:374] T 1cb5db17f1e943b9984b5e458c6ecf76 P 3e21de71b6e9439f80d484412bed0c8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:12:08.209760 16994 raft_consensus.cc:729] T 1cb5db17f1e943b9984b5e458c6ecf76 P 3e21de71b6e9439f80d484412bed0c8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e21de71b6e9439f80d484412bed0c8a, State: Initialized, Role: FOLLOWER
I20260811 19:12:08.210021 16994 consensus_queue.cc:260] T 1cb5db17f1e943b9984b5e458c6ecf76 P 3e21de71b6e9439f80d484412bed0c8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9117d1dafdbe4e6f804a6f7f742596b6" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 34125 } } peers { permanent_uuid: "3e21de71b6e9439f80d484412bed0c8a" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 38449 } } peers { permanent_uuid: "06c375ae44c0466c8b91db22d0b0765c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 32931 } }
I20260811 19:12:08.210868 16994 ts_tablet_manager.cc:1428] T 1cb5db17f1e943b9984b5e458c6ecf76 P 3e21de71b6e9439f80d484412bed0c8a: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 19:12:08.213244 16996 tablet_bootstrap.cc:492] T 1cb5db17f1e943b9984b5e458c6ecf76 P 06c375ae44c0466c8b91db22d0b0765c: Bootstrap starting.
I20260811 19:12:08.215530 16996 tablet_bootstrap.cc:654] T 1cb5db17f1e943b9984b5e458c6ecf76 P 06c375ae44c0466c8b91db22d0b0765c: Neither blocks nor log segments found. Creating new log.
I20260811 19:12:08.216578 17001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:08.212099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33527 (local address 127.3.61.129:34937)
0811 19:12:08.216063 (+ 3964us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:08.216237 (+ 174us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:33527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3801,"thread_start_us":3643,"threads_started":1}
I20260811 19:12:08.217592 17002 tablet_bootstrap.cc:492] T 1cb5db17f1e943b9984b5e458c6ecf76 P 9117d1dafdbe4e6f804a6f7f742596b6: Bootstrap starting.
I20260811 19:12:08.219029 16359 heartbeater.cc:503] Master 127.3.61.188:46509 was elected leader, sending a full tablet report...
I20260811 19:12:08.219134 17001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:08.218610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33527 (local address 127.3.61.129:38259)
0811 19:12:08.218829 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:08.218937 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:33527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260811 19:12:08.219779 17002 tablet_bootstrap.cc:654] T 1cb5db17f1e943b9984b5e458c6ecf76 P 9117d1dafdbe4e6f804a6f7f742596b6: Neither blocks nor log segments found. Creating new log.
I20260811 19:12:08.221772 16996 tablet_bootstrap.cc:492] T 1cb5db17f1e943b9984b5e458c6ecf76 P 06c375ae44c0466c8b91db22d0b0765c: No bootstrap required, opened a new log
I20260811 19:12:08.221938 16996 ts_tablet_manager.cc:1397] T 1cb5db17f1e943b9984b5e458c6ecf76 P 06c375ae44c0466c8b91db22d0b0765c: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260811 19:12:08.222833 17002 tablet_bootstrap.cc:492] T 1cb5db17f1e943b9984b5e458c6ecf76 P 9117d1dafdbe4e6f804a6f7f742596b6: No bootstrap required, opened a new log
I20260811 19:12:08.223009 17002 ts_tablet_manager.cc:1397] T 1cb5db17f1e943b9984b5e458c6ecf76 P 9117d1dafdbe4e6f804a6f7f742596b6: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260811 19:12:08.223198 16996 raft_consensus.cc:348] T 1cb5db17f1e943b9984b5e458c6ecf76 P 06c375ae44c0466c8b91db22d0b0765c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9117d1dafdbe4e6f804a6f7f742596b6" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 34125 } } peers { permanent_uuid: "3e21de71b6e9439f80d484412bed0c8a" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 38449 } } peers { permanent_uuid: "06c375ae44c0466c8b91db22d0b0765c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 32931 } }
I20260811 19:12:08.223488 16996 raft_consensus.cc:374] T 1cb5db17f1e943b9984b5e458c6ecf76 P 06c375ae44c0466c8b91db22d0b0765c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:12:08.223631 16996 raft_consensus.cc:729] T 1cb5db17f1e943b9984b5e458c6ecf76 P 06c375ae44c0466c8b91db22d0b0765c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06c375ae44c0466c8b91db22d0b0765c, State: Initialized, Role: FOLLOWER
I20260811 19:12:08.224184 16996 consensus_queue.cc:260] T 1cb5db17f1e943b9984b5e458c6ecf76 P 06c375ae44c0466c8b91db22d0b0765c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9117d1dafdbe4e6f804a6f7f742596b6" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 34125 } } peers { permanent_uuid: "3e21de71b6e9439f80d484412bed0c8a" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 38449 } } peers { permanent_uuid: "06c375ae44c0466c8b91db22d0b0765c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 32931 } }
I20260811 19:12:08.224143 17002 raft_consensus.cc:348] T 1cb5db17f1e943b9984b5e458c6ecf76 P 9117d1dafdbe4e6f804a6f7f742596b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9117d1dafdbe4e6f804a6f7f742596b6" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 34125 } } peers { permanent_uuid: "3e21de71b6e9439f80d484412bed0c8a" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 38449 } } peers { permanent_uuid: "06c375ae44c0466c8b91db22d0b0765c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 32931 } }
I20260811 19:12:08.224443 17002 raft_consensus.cc:374] T 1cb5db17f1e943b9984b5e458c6ecf76 P 9117d1dafdbe4e6f804a6f7f742596b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:12:08.224525 17002 raft_consensus.cc:729] T 1cb5db17f1e943b9984b5e458c6ecf76 P 9117d1dafdbe4e6f804a6f7f742596b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9117d1dafdbe4e6f804a6f7f742596b6, State: Initialized, Role: FOLLOWER
I20260811 19:12:08.224814 17002 consensus_queue.cc:260] T 1cb5db17f1e943b9984b5e458c6ecf76 P 9117d1dafdbe4e6f804a6f7f742596b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9117d1dafdbe4e6f804a6f7f742596b6" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 34125 } } peers { permanent_uuid: "3e21de71b6e9439f80d484412bed0c8a" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 38449 } } peers { permanent_uuid: "06c375ae44c0466c8b91db22d0b0765c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 32931 } }
I20260811 19:12:08.227306 17011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:08.225884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33527 (local address 127.3.61.130:56513)
0811 19:12:08.226623 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:08.226959 (+ 336us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:33527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":388,"threads_started":1}
I20260811 19:12:08.228245 16996 ts_tablet_manager.cc:1428] T 1cb5db17f1e943b9984b5e458c6ecf76 P 06c375ae44c0466c8b91db22d0b0765c: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260811 19:12:08.228514 16584 heartbeater.cc:503] Master 127.3.61.188:46509 was elected leader, sending a full tablet report...
I20260811 19:12:08.229303 17011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:08.228889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33527 (local address 127.3.61.130:45789)
0811 19:12:08.229065 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:08.229157 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:33527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 19:12:08.231946 17002 ts_tablet_manager.cc:1428] T 1cb5db17f1e943b9984b5e458c6ecf76 P 9117d1dafdbe4e6f804a6f7f742596b6: Time spent starting tablet: real 0.009s user 0.007s sys 0.002s
I20260811 19:12:08.233314 17015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:08.232024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33527 (local address 127.3.61.131:41823)
0811 19:12:08.232788 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:08.232982 (+ 194us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:33527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":479,"threads_started":1}
I20260811 19:12:08.234426 16823 heartbeater.cc:503] Master 127.3.61.188:46509 was elected leader, sending a full tablet report...
I20260811 19:12:08.235180 17015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:08.234532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33527 (local address 127.3.61.131:37971)
0811 19:12:08.234971 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:08.235064 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:33527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172}
I20260811 19:12:08.417372 17010 raft_consensus.cc:482] T 1cb5db17f1e943b9984b5e458c6ecf76 P 9117d1dafdbe4e6f804a6f7f742596b6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:12:08.417732 17010 raft_consensus.cc:504] T 1cb5db17f1e943b9984b5e458c6ecf76 P 9117d1dafdbe4e6f804a6f7f742596b6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9117d1dafdbe4e6f804a6f7f742596b6" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 34125 } } peers { permanent_uuid: "3e21de71b6e9439f80d484412bed0c8a" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 38449 } } peers { permanent_uuid: "06c375ae44c0466c8b91db22d0b0765c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 32931 } }
I20260811 19:12:08.418968 17010 leader_election.cc:288] T 1cb5db17f1e943b9984b5e458c6ecf76 P 9117d1dafdbe4e6f804a6f7f742596b6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e21de71b6e9439f80d484412bed0c8a (127.3.61.129:38449), 06c375ae44c0466c8b91db22d0b0765c (127.3.61.130:32931)
I20260811 19:12:08.419852 16278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1cb5db17f1e943b9984b5e458c6ecf76" candidate_uuid: "9117d1dafdbe4e6f804a6f7f742596b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e21de71b6e9439f80d484412bed0c8a" is_pre_election: true
I20260811 19:12:08.419880 16506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1cb5db17f1e943b9984b5e458c6ecf76" candidate_uuid: "9117d1dafdbe4e6f804a6f7f742596b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06c375ae44c0466c8b91db22d0b0765c" is_pre_election: true
I20260811 19:12:08.420217 16506 raft_consensus.cc:2454] T 1cb5db17f1e943b9984b5e458c6ecf76 P 06c375ae44c0466c8b91db22d0b0765c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9117d1dafdbe4e6f804a6f7f742596b6 in term 0.
I20260811 19:12:08.420215 16278 raft_consensus.cc:2454] T 1cb5db17f1e943b9984b5e458c6ecf76 P 3e21de71b6e9439f80d484412bed0c8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9117d1dafdbe4e6f804a6f7f742596b6 in term 0.
I20260811 19:12:08.420851 16634 leader_election.cc:302] T 1cb5db17f1e943b9984b5e458c6ecf76 P 9117d1dafdbe4e6f804a6f7f742596b6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e21de71b6e9439f80d484412bed0c8a, 9117d1dafdbe4e6f804a6f7f742596b6; no voters:
I20260811 19:12:08.421212 17010 raft_consensus.cc:2781] T 1cb5db17f1e943b9984b5e458c6ecf76 P 9117d1dafdbe4e6f804a6f7f742596b6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:12:08.421489 17010 raft_consensus.cc:482] T 1cb5db17f1e943b9984b5e458c6ecf76 P 9117d1dafdbe4e6f804a6f7f742596b6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:12:08.421638 17010 raft_consensus.cc:3037] T 1cb5db17f1e943b9984b5e458c6ecf76 P 9117d1dafdbe4e6f804a6f7f742596b6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:12:08.423866 17010 raft_consensus.cc:504] T 1cb5db17f1e943b9984b5e458c6ecf76 P 9117d1dafdbe4e6f804a6f7f742596b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9117d1dafdbe4e6f804a6f7f742596b6" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 34125 } } peers { permanent_uuid: "3e21de71b6e9439f80d484412bed0c8a" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 38449 } } peers { permanent_uuid: "06c375ae44c0466c8b91db22d0b0765c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 32931 } }
I20260811 19:12:08.424479 17010 leader_election.cc:288] T 1cb5db17f1e943b9984b5e458c6ecf76 P 9117d1dafdbe4e6f804a6f7f742596b6 [CANDIDATE]: Term 1 election: Requested vote from peers 3e21de71b6e9439f80d484412bed0c8a (127.3.61.129:38449), 06c375ae44c0466c8b91db22d0b0765c (127.3.61.130:32931)
I20260811 19:12:08.425202 16278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1cb5db17f1e943b9984b5e458c6ecf76" candidate_uuid: "9117d1dafdbe4e6f804a6f7f742596b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e21de71b6e9439f80d484412bed0c8a"
I20260811 19:12:08.425273 16506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1cb5db17f1e943b9984b5e458c6ecf76" candidate_uuid: "9117d1dafdbe4e6f804a6f7f742596b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06c375ae44c0466c8b91db22d0b0765c"
I20260811 19:12:08.425752 16278 raft_consensus.cc:3037] T 1cb5db17f1e943b9984b5e458c6ecf76 P 3e21de71b6e9439f80d484412bed0c8a [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:12:08.425817 16506 raft_consensus.cc:3037] T 1cb5db17f1e943b9984b5e458c6ecf76 P 06c375ae44c0466c8b91db22d0b0765c [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:12:08.428666 16278 raft_consensus.cc:2454] T 1cb5db17f1e943b9984b5e458c6ecf76 P 3e21de71b6e9439f80d484412bed0c8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9117d1dafdbe4e6f804a6f7f742596b6 in term 1.
I20260811 19:12:08.428776 16506 raft_consensus.cc:2454] T 1cb5db17f1e943b9984b5e458c6ecf76 P 06c375ae44c0466c8b91db22d0b0765c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9117d1dafdbe4e6f804a6f7f742596b6 in term 1.
I20260811 19:12:08.429337 16631 leader_election.cc:302] T 1cb5db17f1e943b9984b5e458c6ecf76 P 9117d1dafdbe4e6f804a6f7f742596b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 06c375ae44c0466c8b91db22d0b0765c, 3e21de71b6e9439f80d484412bed0c8a, 9117d1dafdbe4e6f804a6f7f742596b6; no voters:
I20260811 19:12:08.429812 17010 raft_consensus.cc:2781] T 1cb5db17f1e943b9984b5e458c6ecf76 P 9117d1dafdbe4e6f804a6f7f742596b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:12:08.430365 17010 raft_consensus.cc:686] T 1cb5db17f1e943b9984b5e458c6ecf76 P 9117d1dafdbe4e6f804a6f7f742596b6 [term 1 LEADER]: Becoming Leader. State: Replica: 9117d1dafdbe4e6f804a6f7f742596b6, State: Running, Role: LEADER
I20260811 19:12:08.430702 17010 consensus_queue.cc:237] T 1cb5db17f1e943b9984b5e458c6ecf76 P 9117d1dafdbe4e6f804a6f7f742596b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9117d1dafdbe4e6f804a6f7f742596b6" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 34125 } } peers { permanent_uuid: "3e21de71b6e9439f80d484412bed0c8a" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 38449 } } peers { permanent_uuid: "06c375ae44c0466c8b91db22d0b0765c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 32931 } }
I20260811 19:12:08.431823 17015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:08.431274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33527 (local address 127.3.61.131:40757)
0811 19:12:08.431486 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:08.431635 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:33527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 19:12:08.434162 16066 catalog_manager.cc:5495] T 1cb5db17f1e943b9984b5e458c6ecf76 P 9117d1dafdbe4e6f804a6f7f742596b6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9117d1dafdbe4e6f804a6f7f742596b6 (127.3.61.131). New cstate: current_term: 1 leader_uuid: "9117d1dafdbe4e6f804a6f7f742596b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9117d1dafdbe4e6f804a6f7f742596b6" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 34125 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e21de71b6e9439f80d484412bed0c8a" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 38449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06c375ae44c0466c8b91db22d0b0765c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 32931 } health_report { overall_health: UNKNOWN } } }
I20260811 19:12:08.458033 16989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:08.452608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36926 (local address 127.3.61.131:34125)
0811 19:12:08.452865 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:08.452870 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:08.452890 (+ 20us) server_negotiation.cc:407] Connection header received
0811 19:12:08.452968 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:08.452972 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:08.453041 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:08.453128 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:12:08.453520 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:08.454296 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:08.455636 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:08.456093 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:08.456206 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:08.456975 (+ 769us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:12:08.457092 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:12:08.457180 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:08.457585 (+ 405us) server_negotiation.cc:299] Negotiation successful
0811 19:12:08.457721 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 19:12:08.462455 16506 raft_consensus.cc:1261] T 1cb5db17f1e943b9984b5e458c6ecf76 P 06c375ae44c0466c8b91db22d0b0765c [term 1 FOLLOWER]: Refusing update from remote peer 9117d1dafdbe4e6f804a6f7f742596b6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:12:08.462456 16278 raft_consensus.cc:1261] T 1cb5db17f1e943b9984b5e458c6ecf76 P 3e21de71b6e9439f80d484412bed0c8a [term 1 FOLLOWER]: Refusing update from remote peer 9117d1dafdbe4e6f804a6f7f742596b6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:12:08.463449 17016 consensus_queue.cc:1035] T 1cb5db17f1e943b9984b5e458c6ecf76 P 9117d1dafdbe4e6f804a6f7f742596b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06c375ae44c0466c8b91db22d0b0765c" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 32931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:12:08.463922 17010 consensus_queue.cc:1035] T 1cb5db17f1e943b9984b5e458c6ecf76 P 9117d1dafdbe4e6f804a6f7f742596b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e21de71b6e9439f80d484412bed0c8a" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 38449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:12:08.468370 17001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:08.467148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33527 (local address 127.3.61.129:46111)
0811 19:12:08.468058 (+ 910us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:08.468175 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:33527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":763}
I20260811 19:12:08.468953 17011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:08.466754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33527 (local address 127.3.61.130:50559)
0811 19:12:08.468648 (+ 1894us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:08.468764 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:33527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1770}
I20260811 19:12:08.481711 17025 mvcc.cc:205] Tried to move back new op lower bound from 7317403764573462528 to 7317403764455985152. Current Snapshot: MvccSnapshot[applied={T|T < 7317403764573462528}]
I20260811 19:12:08.484126 17011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:08.482488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33527 (local address 127.3.61.130:45199)
0811 19:12:08.483803 (+ 1315us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:08.483911 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:33527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1183}
I20260811 19:12:08.484969 17001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:08.482999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33527 (local address 127.3.61.129:48993)
0811 19:12:08.484598 (+ 1599us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:08.484752 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:33527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1316}
I20260811 19:12:08.489524 17015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:08.489038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33527 (local address 127.3.61.131:38433)
0811 19:12:08.489210 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:08.489336 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:33527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 19:12:08.497641 16066 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:12:08.498070 16066 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:12:08.505079 16066 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 006be47d760e41be8683477553cb2bf3: Sending DeleteTablet for 3 replicas of tablet 1cb5db17f1e943b9984b5e458c6ecf76
I20260811 19:12:08.506449 16473 tablet_service.cc:1514] Processing DeleteTablet for tablet 1cb5db17f1e943b9984b5e458c6ecf76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:12:08 UTC) from {username='oryx', principal='oryx/127.3.61.188@KRBTEST.COM'} at 127.0.0.1:34446
I20260811 19:12:08.506449 16712 tablet_service.cc:1514] Processing DeleteTablet for tablet 1cb5db17f1e943b9984b5e458c6ecf76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:12:08 UTC) from {username='oryx', principal='oryx/127.3.61.188@KRBTEST.COM'} at 127.0.0.1:36916
I20260811 19:12:08.506724 16246 tablet_service.cc:1514] Processing DeleteTablet for tablet 1cb5db17f1e943b9984b5e458c6ecf76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:12:08 UTC) from {username='oryx', principal='oryx/127.3.61.188@KRBTEST.COM'} at 127.0.0.1:38772
I20260811 19:12:08.507254 17043 tablet_replica.cc:330] T 1cb5db17f1e943b9984b5e458c6ecf76 P 06c375ae44c0466c8b91db22d0b0765c: stopping tablet replica
I20260811 19:12:08.507463 17043 raft_consensus.cc:2229] T 1cb5db17f1e943b9984b5e458c6ecf76 P 06c375ae44c0466c8b91db22d0b0765c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:12:08.507812 17043 raft_consensus.cc:2258] T 1cb5db17f1e943b9984b5e458c6ecf76 P 06c375ae44c0466c8b91db22d0b0765c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:12:08.508320 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 16025
I20260811 19:12:08.514794 17043 ts_tablet_manager.cc:1905] T 1cb5db17f1e943b9984b5e458c6ecf76 P 06c375ae44c0466c8b91db22d0b0765c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:12:08.523758 17043 ts_tablet_manager.cc:1918] T 1cb5db17f1e943b9984b5e458c6ecf76 P 06c375ae44c0466c8b91db22d0b0765c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:12:08.523959 17043 log.cc:1196] T 1cb5db17f1e943b9984b5e458c6ecf76 P 06c375ae44c0466c8b91db22d0b0765c: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-1/wal/wals/1cb5db17f1e943b9984b5e458c6ecf76
I20260811 19:12:08.524511 17043 ts_tablet_manager.cc:1939] T 1cb5db17f1e943b9984b5e458c6ecf76 P 06c375ae44c0466c8b91db22d0b0765c: Deleting consensus metadata
I20260811 19:12:08.525353 17045 tablet_replica.cc:330] T 1cb5db17f1e943b9984b5e458c6ecf76 P 3e21de71b6e9439f80d484412bed0c8a: stopping tablet replica
I20260811 19:12:08.525586 17045 raft_consensus.cc:2229] T 1cb5db17f1e943b9984b5e458c6ecf76 P 3e21de71b6e9439f80d484412bed0c8a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:12:08.525950 16040 catalog_manager.cc:4841] TS 06c375ae44c0466c8b91db22d0b0765c (127.3.61.130:32931): tablet 1cb5db17f1e943b9984b5e458c6ecf76 (table test-table [id=5a1950f00bbd45f99683baf6f55eb4f3]) successfully deleted
I20260811 19:12:08.526120 17040 tablet_replica.cc:330] T 1cb5db17f1e943b9984b5e458c6ecf76 P 9117d1dafdbe4e6f804a6f7f742596b6: stopping tablet replica
I20260811 19:12:08.526258 17045 raft_consensus.cc:2258] T 1cb5db17f1e943b9984b5e458c6ecf76 P 3e21de71b6e9439f80d484412bed0c8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:12:08.526281 17040 raft_consensus.cc:2229] T 1cb5db17f1e943b9984b5e458c6ecf76 P 9117d1dafdbe4e6f804a6f7f742596b6 [term 1 LEADER]: Raft consensus shutting down.
I20260811 19:12:08.526757 17040 raft_consensus.cc:2258] T 1cb5db17f1e943b9984b5e458c6ecf76 P 9117d1dafdbe4e6f804a6f7f742596b6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:12:08.527490 17045 ts_tablet_manager.cc:1905] T 1cb5db17f1e943b9984b5e458c6ecf76 P 3e21de71b6e9439f80d484412bed0c8a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:12:08.528184 17040 ts_tablet_manager.cc:1905] T 1cb5db17f1e943b9984b5e458c6ecf76 P 9117d1dafdbe4e6f804a6f7f742596b6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:12:08.533588 17040 ts_tablet_manager.cc:1918] T 1cb5db17f1e943b9984b5e458c6ecf76 P 9117d1dafdbe4e6f804a6f7f742596b6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:12:08.533779 17040 log.cc:1196] T 1cb5db17f1e943b9984b5e458c6ecf76 P 9117d1dafdbe4e6f804a6f7f742596b6: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-2/wal/wals/1cb5db17f1e943b9984b5e458c6ecf76
I20260811 19:12:08.534235 17040 ts_tablet_manager.cc:1939] T 1cb5db17f1e943b9984b5e458c6ecf76 P 9117d1dafdbe4e6f804a6f7f742596b6: Deleting consensus metadata
I20260811 19:12:08.534479 17045 ts_tablet_manager.cc:1918] T 1cb5db17f1e943b9984b5e458c6ecf76 P 3e21de71b6e9439f80d484412bed0c8a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:12:08.534631 17045 log.cc:1196] T 1cb5db17f1e943b9984b5e458c6ecf76 P 3e21de71b6e9439f80d484412bed0c8a: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786475454804547-3318-0/minicluster-data/ts-0/wal/wals/1cb5db17f1e943b9984b5e458c6ecf76
I20260811 19:12:08.535135 17045 ts_tablet_manager.cc:1939] T 1cb5db17f1e943b9984b5e458c6ecf76 P 3e21de71b6e9439f80d484412bed0c8a: Deleting consensus metadata
I20260811 19:12:08.535503 16041 catalog_manager.cc:4841] TS 9117d1dafdbe4e6f804a6f7f742596b6 (127.3.61.131:34125): tablet 1cb5db17f1e943b9984b5e458c6ecf76 (table test-table [id=5a1950f00bbd45f99683baf6f55eb4f3]) successfully deleted
I20260811 19:12:08.536403 16042 catalog_manager.cc:4841] TS 3e21de71b6e9439f80d484412bed0c8a (127.3.61.129:38449): tablet 1cb5db17f1e943b9984b5e458c6ecf76 (table test-table [id=5a1950f00bbd45f99683baf6f55eb4f3]) successfully deleted
I20260811 19:12:08.540858 16990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:08.509495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38778 (local address 127.3.61.129:38449)
0811 19:12:08.509767 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:08.509775 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:08.519843 (+ 10068us) server_negotiation.cc:407] Connection header received
0811 19:12:08.520100 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:08.520109 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:08.520259 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:08.520373 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:08.523692 (+ 3319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:08.524663 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:08.527311 (+ 2648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:08.528156 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:08.528309 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:08.532323 (+ 4014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:08.532345 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:08.532388 (+ 43us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:08.532460 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:08.536949 (+ 4489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:08.537943 (+ 994us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:08.537955 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:08.537957 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:08.538226 (+ 269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:08.538744 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:08.538749 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:08.538752 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:08.539394 (+ 642us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:08.539743 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:08.540178 (+ 435us) server_negotiation.cc:299] Negotiation successful
0811 19:12:08.540504 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 19:12:08.541868 16351 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:12:08.562314 16968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:08.561450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33527 (local address 127.0.0.1:49242)
0811 19:12:08.561839 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:08.562008 (+ 169us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:33527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260811 19:12:08.563710 16968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:08.563237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33527 (local address 127.0.0.1:49248)
0811 19:12:08.563433 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:08.563525 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:33527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260811 19:12:08.647408 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 16025
I20260811 19:12:08.672827 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 16373
I20260811 19:12:08.689266 16992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:08.673688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34452 (local address 127.3.61.130:32931)
0811 19:12:08.673862 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:08.673867 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:08.673905 (+ 38us) server_negotiation.cc:407] Connection header received
0811 19:12:08.674147 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:08.674152 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:08.674225 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:08.674316 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:08.675937 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:08.676658 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:08.678176 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:08.678912 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:08.679102 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:08.682226 (+ 3124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:08.682248 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:08.682273 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:08.682324 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:08.686223 (+ 3899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:08.687049 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:08.687057 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:08.687059 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:08.687233 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:08.687680 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:08.687685 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:08.687687 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:08.688091 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:08.688335 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:08.688777 (+ 442us) server_negotiation.cc:299] Negotiation successful
0811 19:12:08.689002 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 19:12:08.689976 16571 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:12:08.794359 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 16373
I20260811 19:12:08.823115 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 16605
I20260811 19:12:08.840441 16989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:08.824245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36940 (local address 127.3.61.131:34125)
0811 19:12:08.824537 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:08.824542 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:08.824568 (+ 26us) server_negotiation.cc:407] Connection header received
0811 19:12:08.824795 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:08.824801 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:08.824896 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:08.825035 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:08.827232 (+ 2197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:08.828110 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:08.829691 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:08.830096 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:08.830209 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:08.833216 (+ 3007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:08.833235 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:08.833256 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:08.833296 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:08.836810 (+ 3514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:08.837792 (+ 982us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:08.837804 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:08.837807 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:08.838098 (+ 291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:08.838611 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:08.838616 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:08.838619 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:08.839074 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:08.839339 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:08.839905 (+ 566us) server_negotiation.cc:299] Negotiation successful
0811 19:12:08.840114 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260811 19:12:08.841136 16813 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:12:08.977583 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 16605
I20260811 19:12:09.013643 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 15519
I20260811 19:12:09.015035 15802 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:12:09.148916 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 15519
I20260811 19:12:09.157881 17053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:09.157034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33527 (local address 127.0.0.1:49254)
0811 19:12:09.157557 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:09.157648 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:33527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":277,"threads_started":1}
I20260811 19:12:09.159744 17053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:09.159190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:33527 (local address 127.0.0.1:49260)
0811 19:12:09.159440 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:09.159564 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:33527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260811 19:12:09.181699 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 15683
I20260811 19:12:09.183203 16112 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:12:09.317386 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 15683
2026-08-11T19:12:09Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (13312 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:12:09 dist-test-slave-t7g5 krb5kdc[17058](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:12:09 dist-test-slave-t7g5 krb5kdc[17058](info): set up 2 sockets
Aug 11 19:12:09 dist-test-slave-t7g5 krb5kdc[17058](info): commencing operation
krb5kdc: starting...
W20260811 19:12:10.661055 3318 mini_kdc.cc:121] Time spent starting KDC: real 1.283s user 0.003s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Aug 11 19:12:10 dist-test-slave-t7g5 krb5kdc[17058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475530, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:12:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:12:10Z Disabled control of system clock
WARNING: no policy specified for oryx/127.3.61.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.61.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.61.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/krb5kdc/oryx_127.3.61.190.keytab.
Entry for principal oryx/127.3.61.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/krb5kdc/oryx_127.3.61.190.keytab.
I20260811 19:12:10.997067 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:45797
--webserver_interface=127.3.61.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/krb5kdc/oryx_127.3.61.190.keytab
--principal=oryx/127.3.61.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:39423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.61.190:45797
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:12:11.390123 17083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:12:11.390489 17083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:12:11.390631 17083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:12:11.398190 17083 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:12:11.398341 17083 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 19:12:11.398415 17083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:12:11.398507 17083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:12:11.398576 17083 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:12:11.398664 17083 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:12:11.410862 17083 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:39423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.61.190:45797
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/krb5kdc/oryx_127.3.61.190.keytab
--principal=oryx/127.3.61.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:45797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.61.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:12:11.412345 17083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:12:11.413775 17083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:12:11.427654 17093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:11.427719 17090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:11.427733 17091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:12:11.429028 17083 server_base.cc:1034] running on GCE node
I20260811 19:12:11.429978 17083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:12:11.431603 17083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:12:11.432910 17083 hybrid_clock.cc:648] HybridClock initialized: now 1786475531432826 us; error 47 us; skew 500 ppm
I20260811 19:12:11.433296 17083 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:12:11 dist-test-slave-t7g5 krb5kdc[17058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475531, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.61.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:12:11.437016 17083 init.cc:377] Logged in from keytab as oryx/127.3.61.190@KRBTEST.COM (short username oryx)
I20260811 19:12:11.439127 17083 webserver.cc:460] Webserver started at http://127.3.61.190:43807/ using document root <none> and password file <none>
I20260811 19:12:11.440425 17083 fs_manager.cc:359] Metadata directory not provided
I20260811 19:12:11.440568 17083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:12:11.441141 17083 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:12:11.444422 17083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/master-0/data/instance:
uuid: "21afe580e560414d97610b190fba81cd"
format_stamp: "Formatted at 2026-08-11 19:12:11 on dist-test-slave-t7g5"
server_key: "e340722587e08483c78909af248d6d63"
server_key_iv: "b69629bd6c98a4d6b2eb2c8d90eab7cc"
server_key_version: "encryptionkey@0"
I20260811 19:12:11.445241 17083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/master-0/wal/instance:
uuid: "21afe580e560414d97610b190fba81cd"
format_stamp: "Formatted at 2026-08-11 19:12:11 on dist-test-slave-t7g5"
server_key: "e340722587e08483c78909af248d6d63"
server_key_iv: "b69629bd6c98a4d6b2eb2c8d90eab7cc"
server_key_version: "encryptionkey@0"
I20260811 19:12:11.450959 17083 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260811 19:12:11.454955 17104 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:11.456738 17083 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260811 19:12:11.457090 17083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/master-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/master-0/wal
uuid: "21afe580e560414d97610b190fba81cd"
format_stamp: "Formatted at 2026-08-11 19:12:11 on dist-test-slave-t7g5"
server_key: "e340722587e08483c78909af248d6d63"
server_key_iv: "b69629bd6c98a4d6b2eb2c8d90eab7cc"
server_key_version: "encryptionkey@0"
I20260811 19:12:11.457360 17083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:12:11.478209 17083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:12:11.484409 17083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:12:11.484869 17083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:12:11.511695 17083 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.190:45797
I20260811 19:12:11.511734 17216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.190:45797 every 8 connection(s)
I20260811 19:12:11.513193 17083 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
I20260811 19:12:11.517553 17218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:12:11.523255 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 17083
I20260811 19:12:11.523546 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/master-0/wal/instance
I20260811 19:12:11.524507 3318 external_mini_cluster.cc:1428] Setting key c96a580fadcaaea9eda323850ea74749
I20260811 19:12:11.533461 17218 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21afe580e560414d97610b190fba81cd: Bootstrap starting.
I20260811 19:12:11.538018 17218 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 21afe580e560414d97610b190fba81cd: Neither blocks nor log segments found. Creating new log.
I20260811 19:12:11.539803 17218 log.cc:824] T 00000000000000000000000000000000 P 21afe580e560414d97610b190fba81cd: Log is configured to *not* fsync() on all Append() calls
Aug 11 19:12:11 dist-test-slave-t7g5 krb5kdc[17058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475530, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM
I20260811 19:12:11.544705 17218 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21afe580e560414d97610b190fba81cd: No bootstrap required, opened a new log
I20260811 19:12:11.553807 17218 raft_consensus.cc:348] T 00000000000000000000000000000000 P 21afe580e560414d97610b190fba81cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21afe580e560414d97610b190fba81cd" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 45797 } }
I20260811 19:12:11.554255 17218 raft_consensus.cc:374] T 00000000000000000000000000000000 P 21afe580e560414d97610b190fba81cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:12:11.554368 17218 raft_consensus.cc:729] T 00000000000000000000000000000000 P 21afe580e560414d97610b190fba81cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21afe580e560414d97610b190fba81cd, State: Initialized, Role: FOLLOWER
I20260811 19:12:11.554728 17224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:11.527602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42422 (local address 127.3.61.190:45797)
0811 19:12:11.528640 (+ 1038us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:11.528654 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:11.528709 (+ 55us) server_negotiation.cc:407] Connection header received
0811 19:12:11.531430 (+ 2721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:11.531463 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:11.531955 (+ 492us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:11.532352 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:11.534405 (+ 2053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:11.536087 (+ 1682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:11.538695 (+ 2608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:11.539819 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:11.540080 (+ 261us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:11.546086 (+ 6006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:11.546098 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:11.546121 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:11.546178 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:11.549707 (+ 3529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:11.550601 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:11.550609 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:11.550617 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:11.550825 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:11.551259 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:11.551263 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:11.551265 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:11.552101 (+ 836us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:11.552465 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:11.552997 (+ 532us) server_negotiation.cc:299] Negotiation successful
0811 19:12:11.553489 (+ 492us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":728,"thread_start_us":453,"threads_started":1}
I20260811 19:12:11.555402 17218 consensus_queue.cc:260] T 00000000000000000000000000000000 P 21afe580e560414d97610b190fba81cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21afe580e560414d97610b190fba81cd" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 45797 } }
I20260811 19:12:11.555735 17218 raft_consensus.cc:388] T 00000000000000000000000000000000 P 21afe580e560414d97610b190fba81cd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:12:11.555878 17228 raft_consensus.cc:482] T 00000000000000000000000000000000 P 21afe580e560414d97610b190fba81cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:12:11.556128 17228 raft_consensus.cc:504] T 00000000000000000000000000000000 P 21afe580e560414d97610b190fba81cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21afe580e560414d97610b190fba81cd" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 45797 } }
I20260811 19:12:11.556734 17218 raft_consensus.cc:482] T 00000000000000000000000000000000 P 21afe580e560414d97610b190fba81cd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:12:11.556888 17218 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 21afe580e560414d97610b190fba81cd [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:12:11.556774 17228 leader_election.cc:302] T 00000000000000000000000000000000 P 21afe580e560414d97610b190fba81cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 21afe580e560414d97610b190fba81cd; no voters:
I20260811 19:12:11.557615 17228 leader_election.cc:288] T 00000000000000000000000000000000 P 21afe580e560414d97610b190fba81cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers
I20260811 19:12:11.559820 17218 raft_consensus.cc:504] T 00000000000000000000000000000000 P 21afe580e560414d97610b190fba81cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21afe580e560414d97610b190fba81cd" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 45797 } }
I20260811 19:12:11.560092 17218 leader_election.cc:302] T 00000000000000000000000000000000 P 21afe580e560414d97610b190fba81cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 21afe580e560414d97610b190fba81cd; no voters:
I20260811 19:12:11.560151 17232 raft_consensus.cc:2741] T 00000000000000000000000000000000 P 21afe580e560414d97610b190fba81cd [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260811 19:12:11.560324 17218 leader_election.cc:288] T 00000000000000000000000000000000 P 21afe580e560414d97610b190fba81cd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:12:11.560693 17228 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 21afe580e560414d97610b190fba81cd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:12:11.561785 17228 raft_consensus.cc:686] T 00000000000000000000000000000000 P 21afe580e560414d97610b190fba81cd [term 1 LEADER]: Becoming Leader. State: Replica: 21afe580e560414d97610b190fba81cd, State: Running, Role: LEADER
I20260811 19:12:11.562602 17228 consensus_queue.cc:237] T 00000000000000000000000000000000 P 21afe580e560414d97610b190fba81cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21afe580e560414d97610b190fba81cd" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 45797 } }
I20260811 19:12:11.563109 17218 sys_catalog.cc:564] T 00000000000000000000000000000000 P 21afe580e560414d97610b190fba81cd [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:12:11.566048 17232 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21afe580e560414d97610b190fba81cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 21afe580e560414d97610b190fba81cd. Latest consensus state: current_term: 1 leader_uuid: "21afe580e560414d97610b190fba81cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21afe580e560414d97610b190fba81cd" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 45797 } } }
I20260811 19:12:11.566577 17232 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21afe580e560414d97610b190fba81cd [sys.catalog]: This master's current role is: LEADER
I20260811 19:12:11.567029 17233 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21afe580e560414d97610b190fba81cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "21afe580e560414d97610b190fba81cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21afe580e560414d97610b190fba81cd" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 45797 } } }
I20260811 19:12:11.567313 17233 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21afe580e560414d97610b190fba81cd [sys.catalog]: This master's current role is: LEADER
I20260811 19:12:11.569705 17244 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:12:11.574540 17244 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:12:11.588397 17244 catalog_manager.cc:1347] Generated new cluster ID: ab50ab8b3f2244cb9f3c07e564df6139
I20260811 19:12:11.588589 17244 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:12:11.605984 17244 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:12:11.608038 17244 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:12:11.620621 17244 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 21afe580e560414d97610b190fba81cd: Generated new TSK 0
I20260811 19:12:11.621922 17244 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.3.61.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.61.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.61.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/krb5kdc/oryx_127.3.61.129.keytab.
Entry for principal oryx/127.3.61.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/krb5kdc/oryx_127.3.61.129.keytab.
I20260811 19:12:11.714404 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.129:0
--local_ip_for_outbound_sockets=127.3.61.129
--webserver_interface=127.3.61.129
--webserver_port=0
--tserver_master_addrs=127.3.61.190:45797
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/krb5kdc/oryx_127.3.61.129.keytab
--principal=oryx/127.3.61.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:39423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:12:12.122464 17270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:12:12.122788 17270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:12:12.122913 17270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:12:12.130993 17270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:12:12.131150 17270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:12:12.131271 17270 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 19:12:12.131397 17270 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.129
I20260811 19:12:12.142344 17270 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:39423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/krb5kdc/oryx_127.3.61.129.keytab
--principal=oryx/127.3.61.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.61.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.61.190:45797
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.129
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:12:12.144400 17270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:12:12.146539 17270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:12:12.162319 17278 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:12.162289 17284 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:12.162387 17282 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:12:12.163228 17270 server_base.cc:1034] running on GCE node
I20260811 19:12:12.163983 17270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:12:12.165118 17270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:12:12.166401 17270 hybrid_clock.cc:648] HybridClock initialized: now 1786475532166310 us; error 68 us; skew 500 ppm
I20260811 19:12:12.166776 17270 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:12:12 dist-test-slave-t7g5 krb5kdc[17058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475532, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.61.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:12:12.170501 17270 init.cc:377] Logged in from keytab as oryx/127.3.61.129@KRBTEST.COM (short username oryx)
I20260811 19:12:12.172199 17270 webserver.cc:460] Webserver started at http://127.3.61.129:36503/ using document root <none> and password file <none>
I20260811 19:12:12.173012 17270 fs_manager.cc:359] Metadata directory not provided
I20260811 19:12:12.173127 17270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:12:12.173478 17270 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:12:12.178167 17270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-0/data/instance:
uuid: "8b314dd688dd4682b73c26337f53a6cd"
format_stamp: "Formatted at 2026-08-11 19:12:12 on dist-test-slave-t7g5"
server_key: "2bc943759a27bb540326af166643090c"
server_key_iv: "e2b08397f9b765f53c2cdc12b890ab73"
server_key_version: "encryptionkey@0"
I20260811 19:12:12.179416 17270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-0/wal/instance:
uuid: "8b314dd688dd4682b73c26337f53a6cd"
format_stamp: "Formatted at 2026-08-11 19:12:12 on dist-test-slave-t7g5"
server_key: "2bc943759a27bb540326af166643090c"
server_key_iv: "e2b08397f9b765f53c2cdc12b890ab73"
server_key_version: "encryptionkey@0"
I20260811 19:12:12.187044 17270 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260811 19:12:12.192123 17296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:12.194324 17270 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 19:12:12.194559 17270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-0/wal
uuid: "8b314dd688dd4682b73c26337f53a6cd"
format_stamp: "Formatted at 2026-08-11 19:12:12 on dist-test-slave-t7g5"
server_key: "2bc943759a27bb540326af166643090c"
server_key_iv: "e2b08397f9b765f53c2cdc12b890ab73"
server_key_version: "encryptionkey@0"
I20260811 19:12:12.194880 17270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:12:12.220574 17270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:12:12.225390 17270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:12:12.225725 17270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:12:12.228693 17270 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:12:12.228792 17270 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:12.228955 17270 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:12:12.229020 17270 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 19:12:12 dist-test-slave-t7g5 krb5kdc[17058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475532, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.61.129@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM
I20260811 19:12:12.257521 17311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:12.229569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:39747 (local address 127.3.61.190:45797)
0811 19:12:12.230092 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:12.230099 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:12.231545 (+ 1446us) server_negotiation.cc:407] Connection header received
0811 19:12:12.235361 (+ 3816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:12.235371 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:12.235555 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:12.235710 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:12.239242 (+ 3532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:12.240469 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:12.242884 (+ 2415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:12.243491 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:12.243650 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:12.250531 (+ 6881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:12.250539 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:12.250544 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:12.250596 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:12.253286 (+ 2690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:12.254427 (+ 1141us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:12.254432 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:12.254434 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:12.254579 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:12.255169 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:12.255175 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:12.255178 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:12.255624 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:12.255802 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:12.256877 (+ 1075us) server_negotiation.cc:299] Negotiation successful
0811 19:12:12.257143 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":260,"threads_started":1}
I20260811 19:12:12.258906 17312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:12.230329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:45797 (local address 127.3.61.129:39747)
0811 19:12:12.231310 (+ 981us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:12.231381 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:12.234673 (+ 3292us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:12.236227 (+ 1554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:12.236249 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:12.237372 (+ 1123us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:12.238852 (+ 1480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:12.238908 (+ 56us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:12.240689 (+ 1781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:12.240695 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:12.242694 (+ 1999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:12.242713 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:12.243832 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:12.243841 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:12.244234 (+ 393us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:12.245220 (+ 986us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:12.245272 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:12.250076 (+ 4804us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:12:12.253481 (+ 3405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:12.253494 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:12.253512 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:12.254265 (+ 753us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:12.254703 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:12.254708 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:12.254710 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:12.254986 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:12.255786 (+ 800us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:12:12.255792 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:12:12.256298 (+ 506us) client_negotiation.cc:769] Sending connection context
0811 19:12:12.256732 (+ 434us) client_negotiation.cc:240] Negotiation successful
0811 19:12:12.257376 (+ 644us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":650,"thread_start_us":396,"threads_started":1}
I20260811 19:12:12.282940 17270 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.129:45885
I20260811 19:12:12.282953 17477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.129:45885 every 8 connection(s)
I20260811 19:12:12.284614 17270 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
I20260811 19:12:12.287279 17478 heartbeater.cc:347] Connected to a master server at 127.3.61.190:45797
I20260811 19:12:12.287791 17478 heartbeater.cc:464] Registering TS with master...
I20260811 19:12:12.288905 17478 heartbeater.cc:511] Master 127.3.61.190:45797 requested a full tablet report, sending...
I20260811 19:12:12.289480 17311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:12.269622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:58433 (local address 127.3.61.190:45797)
0811 19:12:12.269952 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:12.269961 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:12.271522 (+ 1561us) server_negotiation.cc:407] Connection header received
0811 19:12:12.271747 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:12.271753 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:12.271870 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:12.272056 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:12.273404 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:12.274451 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:12.278712 (+ 4261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:12.279449 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:12.279594 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:12.282566 (+ 2972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:12.282573 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:12.282578 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:12.282622 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:12.286007 (+ 3385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:12.286876 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:12.286880 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:12.286882 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:12.287040 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:12.287808 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:12.287812 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:12.287814 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:12.288153 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:12.288297 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:12.288954 (+ 657us) server_negotiation.cc:299] Negotiation successful
0811 19:12:12.289199 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138}
I20260811 19:12:12.289688 17438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:12.270362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:45797 (local address 127.3.61.129:58433)
0811 19:12:12.271350 (+ 988us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:12.271377 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:12.271588 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:12.272146 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:12.272154 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:12.272876 (+ 722us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:12.273210 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:12.273222 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:12.276258 (+ 3036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:12.276266 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:12.278348 (+ 2082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:12.278379 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:12.279953 (+ 1574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:12.279962 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:12.280194 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:12.281083 (+ 889us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:12.281104 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:12.282260 (+ 1156us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:12:12.286191 (+ 3931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:12.286196 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:12.286200 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:12.286736 (+ 536us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:12.287301 (+ 565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:12.287309 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:12.287314 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:12.287602 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:12.288339 (+ 737us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:12:12.288345 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:12:12.288615 (+ 270us) client_negotiation.cc:769] Sending connection context
0811 19:12:12.288800 (+ 185us) client_negotiation.cc:240] Negotiation successful
0811 19:12:12.289184 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":791,"thread_start_us":638,"threads_started":1}
I20260811 19:12:12.291883 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 17270
I20260811 19:12:12.292091 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-0/wal/instance
I20260811 19:12:12.292239 17133 ts_manager.cc:194] Registered new tserver with Master: 8b314dd688dd4682b73c26337f53a6cd (127.3.61.129:45885)
I20260811 19:12:12.292837 3318 external_mini_cluster.cc:1428] Setting key 01e3695fb00d917e290c853c4c692326
I20260811 19:12:12.294955 17133 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.61.129@KRBTEST.COM'} at 127.3.61.129:39747
WARNING: no policy specified for oryx/127.3.61.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.61.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.61.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/krb5kdc/oryx_127.3.61.130.keytab.
Entry for principal oryx/127.3.61.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/krb5kdc/oryx_127.3.61.130.keytab.
I20260811 19:12:12.365522 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.130:0
--local_ip_for_outbound_sockets=127.3.61.130
--webserver_interface=127.3.61.130
--webserver_port=0
--tserver_master_addrs=127.3.61.190:45797
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/krb5kdc/oryx_127.3.61.130.keytab
--principal=oryx/127.3.61.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:39423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:12:12.739553 17482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:12:12.739883 17482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:12:12.740006 17482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:12:12.747629 17482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:12:12.747776 17482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:12:12.747881 17482 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 19:12:12.748014 17482 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.130
I20260811 19:12:12.758611 17482 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:39423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/krb5kdc/oryx_127.3.61.130.keytab
--principal=oryx/127.3.61.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.61.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.61.190:45797
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.130
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:12:12.760118 17482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:12:12.761590 17482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:12:12.776854 17492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:12.776930 17490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:12.776971 17494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:12:12.777733 17482 server_base.cc:1034] running on GCE node
I20260811 19:12:12.778465 17482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:12:12.779503 17482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:12:12.780771 17482 hybrid_clock.cc:648] HybridClock initialized: now 1786475532780673 us; error 65 us; skew 500 ppm
I20260811 19:12:12.781145 17482 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:12:12 dist-test-slave-t7g5 krb5kdc[17058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475532, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.61.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:12:12.784953 17482 init.cc:377] Logged in from keytab as oryx/127.3.61.130@KRBTEST.COM (short username oryx)
I20260811 19:12:12.786617 17482 webserver.cc:460] Webserver started at http://127.3.61.130:33235/ using document root <none> and password file <none>
I20260811 19:12:12.787837 17482 fs_manager.cc:359] Metadata directory not provided
I20260811 19:12:12.788012 17482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:12:12.788582 17482 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:12:12.792910 17482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-1/data/instance:
uuid: "7f82b1726e1d4ee7849c8692b83c0035"
format_stamp: "Formatted at 2026-08-11 19:12:12 on dist-test-slave-t7g5"
server_key: "5299aa4d0650cef1b27886da73187219"
server_key_iv: "c5618e99e21285dd0e816a4a432ef3d5"
server_key_version: "encryptionkey@0"
I20260811 19:12:12.793965 17482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-1/wal/instance:
uuid: "7f82b1726e1d4ee7849c8692b83c0035"
format_stamp: "Formatted at 2026-08-11 19:12:12 on dist-test-slave-t7g5"
server_key: "5299aa4d0650cef1b27886da73187219"
server_key_iv: "c5618e99e21285dd0e816a4a432ef3d5"
server_key_version: "encryptionkey@0"
I20260811 19:12:12.800993 17482 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260811 19:12:12.805380 17508 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:12.807278 17482 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260811 19:12:12.807492 17482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-1/wal
uuid: "7f82b1726e1d4ee7849c8692b83c0035"
format_stamp: "Formatted at 2026-08-11 19:12:12 on dist-test-slave-t7g5"
server_key: "5299aa4d0650cef1b27886da73187219"
server_key_iv: "c5618e99e21285dd0e816a4a432ef3d5"
server_key_version: "encryptionkey@0"
I20260811 19:12:12.807770 17482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:12:12.824874 17482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:12:12.831354 17482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:12:12.831694 17482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:12:12.835068 17482 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:12:12.835191 17482 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:12.835350 17482 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:12:12.835429 17482 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 19:12:12 dist-test-slave-t7g5 krb5kdc[17058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475532, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.61.130@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM
I20260811 19:12:12.860917 17533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:12.836006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:49669 (local address 127.3.61.190:45797)
0811 19:12:12.836652 (+ 646us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:12.836658 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:12.838074 (+ 1416us) server_negotiation.cc:407] Connection header received
0811 19:12:12.840094 (+ 2020us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:12.840100 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:12.840199 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:12.840298 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:12.843086 (+ 2788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:12.844371 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:12.846875 (+ 2504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:12.847483 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:12.847634 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:12.853239 (+ 5605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:12.853246 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:12.853251 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:12.853305 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:12.856912 (+ 3607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:12.858157 (+ 1245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:12.858162 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:12.858164 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:12.858309 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:12.858939 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:12.858945 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:12.858948 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:12.859273 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:12.859432 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:12.860261 (+ 829us) server_negotiation.cc:299] Negotiation successful
0811 19:12:12.860517 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":332,"threads_started":1}
I20260811 19:12:12.862473 17534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:12.836609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:45797 (local address 127.3.61.130:49669)
0811 19:12:12.837730 (+ 1121us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:12.837775 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:12.839770 (+ 1995us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:12.840636 (+ 866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:12.840652 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:12.841643 (+ 991us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:12.842765 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:12.842796 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:12.844580 (+ 1784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:12.844587 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:12.846654 (+ 2067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:12.846694 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:12.847770 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:12.847779 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:12.848014 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:12.848840 (+ 826us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:12.848879 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:12.852835 (+ 3956us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:12:12.857103 (+ 4268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:12.857119 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:12.857136 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:12.857959 (+ 823us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:12.858452 (+ 493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:12.858461 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:12.858464 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:12.858803 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:12.859424 (+ 621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:12:12.859433 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:12:12.859841 (+ 408us) client_negotiation.cc:769] Sending connection context
0811 19:12:12.860112 (+ 271us) client_negotiation.cc:240] Negotiation successful
0811 19:12:12.860705 (+ 593us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":749,"thread_start_us":436,"threads_started":1}
I20260811 19:12:12.884508 17482 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.130:45219
I20260811 19:12:12.884521 17706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.130:45219 every 8 connection(s)
I20260811 19:12:12.885926 17482 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
I20260811 19:12:12.888726 17708 heartbeater.cc:347] Connected to a master server at 127.3.61.190:45797
I20260811 19:12:12.889008 17708 heartbeater.cc:464] Registering TS with master...
I20260811 19:12:12.889061 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 17482
I20260811 19:12:12.889266 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-1/wal/instance
I20260811 19:12:12.889902 3318 external_mini_cluster.cc:1428] Setting key 78b380672c7ae4db9852acf059325833
I20260811 19:12:12.890002 17708 heartbeater.cc:511] Master 127.3.61.190:45797 requested a full tablet report, sending...
I20260811 19:12:12.890251 17533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:12.871830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:51989 (local address 127.3.61.190:45797)
0811 19:12:12.872067 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:12.872074 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:12.873065 (+ 991us) server_negotiation.cc:407] Connection header received
0811 19:12:12.873357 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:12.873364 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:12.873465 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:12.873549 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:12.875407 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:12.876214 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:12.878770 (+ 2556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:12.879397 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:12.879562 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:12.882880 (+ 3318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:12.882885 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:12.882889 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:12.882925 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:12.886046 (+ 3121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:12.887367 (+ 1321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:12.887373 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:12.887377 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:12.887534 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:12.888010 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:12.888015 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:12.888018 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:12.888437 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:12.888606 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:12.889519 (+ 913us) server_negotiation.cc:299] Negotiation successful
0811 19:12:12.889809 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260811 19:12:12.890296 17653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:12.871799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:45797 (local address 127.3.61.130:51989)
0811 19:12:12.872908 (+ 1109us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:12.872935 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:12.873154 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:12.873855 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:12.873860 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:12.874666 (+ 806us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:12.875225 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:12.875243 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:12.876410 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:12.876417 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:12.878553 (+ 2136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:12.878576 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:12.879862 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:12.879870 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:12.880072 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:12.880873 (+ 801us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:12.880894 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:12.882627 (+ 1733us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:12:12.886189 (+ 3562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:12.886195 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:12.886200 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:12.887099 (+ 899us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:12.887679 (+ 580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:12.887685 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:12.887689 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:12.887861 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:12.888836 (+ 975us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:12:12.888841 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:12:12.889110 (+ 269us) client_negotiation.cc:769] Sending connection context
0811 19:12:12.889316 (+ 206us) client_negotiation.cc:240] Negotiation successful
0811 19:12:12.889747 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":897,"thread_start_us":507,"threads_started":1}
I20260811 19:12:12.891817 17131 ts_manager.cc:194] Registered new tserver with Master: 7f82b1726e1d4ee7849c8692b83c0035 (127.3.61.130:45219)
I20260811 19:12:12.892831 17131 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.61.130@KRBTEST.COM'} at 127.3.61.130:49669
WARNING: no policy specified for oryx/127.3.61.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.61.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.61.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/krb5kdc/oryx_127.3.61.131.keytab.
Entry for principal oryx/127.3.61.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/krb5kdc/oryx_127.3.61.131.keytab.
I20260811 19:12:12.962204 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.131:0
--local_ip_for_outbound_sockets=127.3.61.131
--webserver_interface=127.3.61.131
--webserver_port=0
--tserver_master_addrs=127.3.61.190:45797
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/krb5kdc/oryx_127.3.61.131.keytab
--principal=oryx/127.3.61.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:39423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:12:13.299283 17478 heartbeater.cc:503] Master 127.3.61.190:45797 was elected leader, sending a full tablet report...
W20260811 19:12:13.347568 17713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:12:13.347942 17713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:12:13.348091 17713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:12:13.355966 17713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:12:13.356146 17713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:12:13.356277 17713 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 19:12:13.356405 17713 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.131
I20260811 19:12:13.369802 17713 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:39423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/krb5kdc/oryx_127.3.61.131.keytab
--principal=oryx/127.3.61.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.61.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.61.190:45797
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.131
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:12:13.371362 17713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:12:13.372836 17713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:12:13.388749 17719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:13.388720 17723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:13.388861 17721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:12:13.390771 17713 server_base.cc:1034] running on GCE node
I20260811 19:12:13.391691 17713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:12:13.392891 17713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:12:13.394148 17713 hybrid_clock.cc:648] HybridClock initialized: now 1786475533394071 us; error 70 us; skew 500 ppm
I20260811 19:12:13.394503 17713 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:12:13 dist-test-slave-t7g5 krb5kdc[17058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475533, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.61.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:12:13.403312 17713 init.cc:377] Logged in from keytab as oryx/127.3.61.131@KRBTEST.COM (short username oryx)
I20260811 19:12:13.405205 17713 webserver.cc:460] Webserver started at http://127.3.61.131:45501/ using document root <none> and password file <none>
I20260811 19:12:13.406131 17713 fs_manager.cc:359] Metadata directory not provided
I20260811 19:12:13.406250 17713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:12:13.406597 17713 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:12:13.410022 17713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-2/data/instance:
uuid: "2ea91336741c42ae9f80f61152e39ad9"
format_stamp: "Formatted at 2026-08-11 19:12:13 on dist-test-slave-t7g5"
server_key: "bf69ef029888e89f8d3a44207b744a69"
server_key_iv: "5376129358aa0c5682c38958c11a2e15"
server_key_version: "encryptionkey@0"
I20260811 19:12:13.410948 17713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-2/wal/instance:
uuid: "2ea91336741c42ae9f80f61152e39ad9"
format_stamp: "Formatted at 2026-08-11 19:12:13 on dist-test-slave-t7g5"
server_key: "bf69ef029888e89f8d3a44207b744a69"
server_key_iv: "5376129358aa0c5682c38958c11a2e15"
server_key_version: "encryptionkey@0"
I20260811 19:12:13.417587 17713 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260811 19:12:13.423081 17733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:13.425405 17713 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.000s
I20260811 19:12:13.425724 17713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-2/wal
uuid: "2ea91336741c42ae9f80f61152e39ad9"
format_stamp: "Formatted at 2026-08-11 19:12:13 on dist-test-slave-t7g5"
server_key: "bf69ef029888e89f8d3a44207b744a69"
server_key_iv: "5376129358aa0c5682c38958c11a2e15"
server_key_version: "encryptionkey@0"
I20260811 19:12:13.426050 17713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:12:13.461015 17713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:12:13.466885 17713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:12:13.467239 17713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:12:13.470782 17713 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:12:13.470916 17713 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:13.471084 17713 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:12:13.471161 17713 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 19:12:13 dist-test-slave-t7g5 krb5kdc[17058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475533, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.61.131@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM
I20260811 19:12:13.504040 17751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:13.471947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:34225 (local address 127.3.61.190:45797)
0811 19:12:13.472411 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:13.472417 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:13.473726 (+ 1309us) server_negotiation.cc:407] Connection header received
0811 19:12:13.476098 (+ 2372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:13.476105 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:13.476230 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:13.476366 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:13.479175 (+ 2809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:13.480708 (+ 1533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:13.483859 (+ 3151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:13.484539 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:13.484657 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:13.494621 (+ 9964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:13.494631 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:13.494637 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:13.494686 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:13.498933 (+ 4247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:13.500315 (+ 1382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:13.500322 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:13.500326 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:13.500556 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:13.501208 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:13.501213 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:13.501216 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:13.501683 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:13.501858 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:13.503155 (+ 1297us) server_negotiation.cc:299] Negotiation successful
0811 19:12:13.503456 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":233,"threads_started":1}
I20260811 19:12:13.506637 17752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:13.472507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:45797 (local address 127.3.61.131:34225)
0811 19:12:13.473516 (+ 1009us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:13.473566 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:13.475756 (+ 2190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:13.476899 (+ 1143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:13.476912 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:13.478172 (+ 1260us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:13.478943 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:13.478965 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:13.481008 (+ 2043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:13.481015 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:13.483607 (+ 2592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:13.483632 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:13.485106 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:13.485114 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:13.485373 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:13.490125 (+ 4752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:13.490155 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:13.494206 (+ 4051us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:12:13.499184 (+ 4978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:13.499197 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:13.499221 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:13.500097 (+ 876us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:13.500710 (+ 613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:13.500715 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:13.500719 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:13.501037 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:13.501865 (+ 828us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:12:13.501878 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:12:13.502549 (+ 671us) client_negotiation.cc:769] Sending connection context
0811 19:12:13.503005 (+ 456us) client_negotiation.cc:240] Negotiation successful
0811 19:12:13.503780 (+ 775us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":643,"thread_start_us":399,"threads_started":1}
I20260811 19:12:13.531476 17713 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.131:36703
I20260811 19:12:13.531643 17935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.131:36703 every 8 connection(s)
I20260811 19:12:13.533012 17713 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
I20260811 19:12:13.535650 17936 heartbeater.cc:347] Connected to a master server at 127.3.61.190:45797
I20260811 19:12:13.535928 17936 heartbeater.cc:464] Registering TS with master...
I20260811 19:12:13.536938 17936 heartbeater.cc:511] Master 127.3.61.190:45797 requested a full tablet report, sending...
I20260811 19:12:13.538628 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 17713
I20260811 19:12:13.538779 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-2/wal/instance
I20260811 19:12:13.538722 17131 ts_manager.cc:194] Registered new tserver with Master: 2ea91336741c42ae9f80f61152e39ad9 (127.3.61.131:36703)
I20260811 19:12:13.539232 3318 external_mini_cluster.cc:1428] Setting key 9543c528b2a2c2b5a7106e0a515e6043
I20260811 19:12:13.540019 17131 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.61.131@KRBTEST.COM'} at 127.3.61.131:34225
I20260811 19:12:13.540606 17890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:13.521339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:45797 (local address 127.3.61.131:48263)
0811 19:12:13.522060 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:13.522087 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:13.522346 (+ 259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:13.522970 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:13.522974 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:13.523976 (+ 1002us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:13.524626 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:13.524642 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:13.526194 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:13.526203 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:13.528005 (+ 1802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:13.528026 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:13.529288 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:13.529299 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:13.529628 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:13.530532 (+ 904us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:13.530555 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:13.532317 (+ 1762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:12:13.536763 (+ 4446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:13.536768 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:13.536771 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:13.537453 (+ 682us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:13.538255 (+ 802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:13.538262 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:13.538266 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:13.538474 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:13.539271 (+ 797us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:12:13.539277 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:12:13.539520 (+ 243us) client_negotiation.cc:769] Sending connection context
0811 19:12:13.539696 (+ 176us) client_negotiation.cc:240] Negotiation successful
0811 19:12:13.540151 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":415,"threads_started":1}
I20260811 19:12:13.540727 17751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:13.521308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:48263 (local address 127.3.61.190:45797)
0811 19:12:13.521521 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:13.521527 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:13.522196 (+ 669us) server_negotiation.cc:407] Connection header received
0811 19:12:13.522523 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:13.522529 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:13.522644 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:13.522719 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:13.524823 (+ 2104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:13.526015 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:13.528218 (+ 2203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:13.528818 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:13.528963 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:13.532558 (+ 3595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:13.532564 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:13.532567 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:13.532607 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:13.536567 (+ 3960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:13.537788 (+ 1221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:13.537796 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:13.537800 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:13.538093 (+ 293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:13.538619 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:13.538625 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:13.538628 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:13.539095 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:13.539266 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:13.539839 (+ 573us) server_negotiation.cc:299] Negotiation successful
0811 19:12:13.540176 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 19:12:13.544036 3318 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 19:12:13 dist-test-slave-t7g5 krb5kdc[17058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475533, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 19:12:13 dist-test-slave-t7g5 krb5kdc[17058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:12:13 dist-test-slave-t7g5 krb5kdc[17058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 19:12:13.617450 17751 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:12:13.605230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42438 (local address 127.3.61.190:45797)
0811 19:12:13.605567 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:13.605575 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:13.605863 (+ 288us) server_negotiation.cc:407] Connection header received
0811 19:12:13.606123 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:13.606127 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:13.606250 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:13.606352 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:13.608067 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:13.608888 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:13.610581 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:13.611074 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:13.611210 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:13.617092 (+ 5882us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42438: BlockingRecv error: recv got EOF from 127.0.0.1:42438 (error 108)
Metrics: {"server-negotiator.queue_time_us":156}
Aug 11 19:12:13 dist-test-slave-t7g5 krb5kdc[17058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475533, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM
I20260811 19:12:13.637643 17751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:13.622136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42444 (local address 127.3.61.190:45797)
0811 19:12:13.622480 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:13.622487 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:13.622929 (+ 442us) server_negotiation.cc:407] Connection header received
0811 19:12:13.623069 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:13.623074 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:13.623167 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:13.623291 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:13.624795 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:13.625638 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:13.627088 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:13.627585 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:13.627708 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:13.631855 (+ 4147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:13.631861 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:13.631864 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:13.631906 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:13.634731 (+ 2825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:13.635505 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:13.635509 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:13.635511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:13.635647 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:13.636084 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:13.636088 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:13.636090 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:13.636432 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:13.636666 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:13.637027 (+ 361us) server_negotiation.cc:299] Negotiation successful
0811 19:12:13.637242 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260811 19:12:13.643209 17135 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42444:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 19:12:13.650918 17135 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:12:13.678462 17967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:13.665940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59546 (local address 127.3.61.130:45219)
0811 19:12:13.666727 (+ 787us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:13.666737 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:13.667641 (+ 904us) server_negotiation.cc:407] Connection header received
0811 19:12:13.667955 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:13.667965 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:13.668284 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:13.669345 (+ 1061us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:12:13.669582 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:13.671244 (+ 1662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:13.674524 (+ 3280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:13.676149 (+ 1625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:13.677542 (+ 1393us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:13.678025 (+ 483us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:13.678102 (+ 77us) server_negotiation.cc:299] Negotiation successful
0811 19:12:13.678242 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":357,"threads_started":1}
I20260811 19:12:13.678779 17964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:13.665588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.130:45219 (local address 127.0.0.1:59546)
0811 19:12:13.667437 (+ 1849us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:13.667486 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:13.667718 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:13.668596 (+ 878us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:13.668605 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:13.668637 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:12:13.669108 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:13.669123 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:13.671444 (+ 2321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:13.671451 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:13.674223 (+ 2772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:13.674252 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:13.676426 (+ 2174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:13.676434 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:13.676726 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:13.676792 (+ 66us) client_negotiation.cc:769] Sending connection context
0811 19:12:13.676921 (+ 129us) client_negotiation.cc:240] Negotiation successful
0811 19:12:13.677152 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1538,"thread_start_us":1364,"threads_started":1}
I20260811 19:12:13.683044 17597 tablet_service.cc:1467] Processing CreateTablet for tablet bbb8f840c6fa4dbda17bee1e22b6088a (DEFAULT_TABLE table=test-table [id=44d4c5eb8ab644099b900dca4dbf1f00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:12:13.684477 17597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbb8f840c6fa4dbda17bee1e22b6088a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:12:13.688519 17974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:13.670356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54366 (local address 127.3.61.131:36703)
0811 19:12:13.671733 (+ 1377us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:13.671742 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:13.679492 (+ 7750us) server_negotiation.cc:407] Connection header received
0811 19:12:13.679610 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:13.679618 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:13.679880 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:13.680084 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:12:13.680973 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:13.682632 (+ 1659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:13.685072 (+ 2440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:13.686785 (+ 1713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:13.686929 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:13.687261 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:13.687901 (+ 640us) server_negotiation.cc:299] Negotiation successful
0811 19:12:13.688140 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1204,"thread_start_us":1046,"threads_started":1}
I20260811 19:12:13.688581 17970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:13.669851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.131:36703 (local address 127.0.0.1:54366)
0811 19:12:13.679327 (+ 9476us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:13.679352 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:13.679458 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:13.680406 (+ 948us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:13.680411 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:13.680435 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:12:13.680769 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:13.680779 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:13.682900 (+ 2121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:13.682908 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:13.684798 (+ 1890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:13.684816 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:13.687407 (+ 2591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:13.687413 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:13.687620 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:13.687659 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 19:12:13.687816 (+ 157us) client_negotiation.cc:240] Negotiation successful
0811 19:12:13.688101 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9349,"thread_start_us":6475,"threads_started":1}
I20260811 19:12:13.692466 17969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:13.666572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41170 (local address 127.3.61.129:45885)
0811 19:12:13.667319 (+ 747us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:13.667345 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:13.676418 (+ 9073us) server_negotiation.cc:407] Connection header received
0811 19:12:13.676563 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:13.676572 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:13.676817 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:13.677039 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:12:13.684109 (+ 7070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:13.686717 (+ 2608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:13.689751 (+ 3034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:13.691273 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:13.691450 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:13.691674 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:13.691983 (+ 309us) server_negotiation.cc:299] Negotiation successful
0811 19:12:13.692155 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":502,"threads_started":1}
I20260811 19:12:13.692456 17973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:13.665957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:45885 (local address 127.0.0.1:41170)
0811 19:12:13.676087 (+ 10130us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:13.676107 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:13.676243 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:13.677038 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:13.677043 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:13.677069 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:12:13.679104 (+ 2035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:13.679121 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:13.686887 (+ 7766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:13.686893 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:13.689516 (+ 2623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:13.689537 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:13.691553 (+ 2016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:13.691561 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:13.691755 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:13.691793 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 19:12:13.691906 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 19:12:13.692081 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9985,"thread_start_us":13034,"threads_started":1}
I20260811 19:12:13.692883 17824 tablet_service.cc:1467] Processing CreateTablet for tablet bbb8f840c6fa4dbda17bee1e22b6088a (DEFAULT_TABLE table=test-table [id=44d4c5eb8ab644099b900dca4dbf1f00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:12:13.695541 17824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbb8f840c6fa4dbda17bee1e22b6088a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:12:13.696476 17375 tablet_service.cc:1467] Processing CreateTablet for tablet bbb8f840c6fa4dbda17bee1e22b6088a (DEFAULT_TABLE table=test-table [id=44d4c5eb8ab644099b900dca4dbf1f00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:12:13.698072 17375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbb8f840c6fa4dbda17bee1e22b6088a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:12:13.699048 17975 tablet_bootstrap.cc:492] T bbb8f840c6fa4dbda17bee1e22b6088a P 7f82b1726e1d4ee7849c8692b83c0035: Bootstrap starting.
I20260811 19:12:13.703058 17975 tablet_bootstrap.cc:654] T bbb8f840c6fa4dbda17bee1e22b6088a P 7f82b1726e1d4ee7849c8692b83c0035: Neither blocks nor log segments found. Creating new log.
I20260811 19:12:13.704540 17975 log.cc:824] T bbb8f840c6fa4dbda17bee1e22b6088a P 7f82b1726e1d4ee7849c8692b83c0035: Log is configured to *not* fsync() on all Append() calls
I20260811 19:12:13.708925 17975 tablet_bootstrap.cc:492] T bbb8f840c6fa4dbda17bee1e22b6088a P 7f82b1726e1d4ee7849c8692b83c0035: No bootstrap required, opened a new log
I20260811 19:12:13.709345 17975 ts_tablet_manager.cc:1397] T bbb8f840c6fa4dbda17bee1e22b6088a P 7f82b1726e1d4ee7849c8692b83c0035: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260811 19:12:13.714902 17978 tablet_bootstrap.cc:492] T bbb8f840c6fa4dbda17bee1e22b6088a P 8b314dd688dd4682b73c26337f53a6cd: Bootstrap starting.
I20260811 19:12:13.716941 17977 tablet_bootstrap.cc:492] T bbb8f840c6fa4dbda17bee1e22b6088a P 2ea91336741c42ae9f80f61152e39ad9: Bootstrap starting.
I20260811 19:12:13.719400 17978 tablet_bootstrap.cc:654] T bbb8f840c6fa4dbda17bee1e22b6088a P 8b314dd688dd4682b73c26337f53a6cd: Neither blocks nor log segments found. Creating new log.
I20260811 19:12:13.720701 17977 tablet_bootstrap.cc:654] T bbb8f840c6fa4dbda17bee1e22b6088a P 2ea91336741c42ae9f80f61152e39ad9: Neither blocks nor log segments found. Creating new log.
I20260811 19:12:13.720875 17978 log.cc:824] T bbb8f840c6fa4dbda17bee1e22b6088a P 8b314dd688dd4682b73c26337f53a6cd: Log is configured to *not* fsync() on all Append() calls
I20260811 19:12:13.721249 17975 raft_consensus.cc:348] T bbb8f840c6fa4dbda17bee1e22b6088a P 7f82b1726e1d4ee7849c8692b83c0035 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 } } peers { permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 } } peers { permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 } }
I20260811 19:12:13.721742 17975 raft_consensus.cc:374] T bbb8f840c6fa4dbda17bee1e22b6088a P 7f82b1726e1d4ee7849c8692b83c0035 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:12:13.721861 17975 raft_consensus.cc:729] T bbb8f840c6fa4dbda17bee1e22b6088a P 7f82b1726e1d4ee7849c8692b83c0035 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f82b1726e1d4ee7849c8692b83c0035, State: Initialized, Role: FOLLOWER
I20260811 19:12:13.722018 17977 log.cc:824] T bbb8f840c6fa4dbda17bee1e22b6088a P 2ea91336741c42ae9f80f61152e39ad9: Log is configured to *not* fsync() on all Append() calls
I20260811 19:12:13.722662 17975 consensus_queue.cc:260] T bbb8f840c6fa4dbda17bee1e22b6088a P 7f82b1726e1d4ee7849c8692b83c0035 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 } } peers { permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 } } peers { permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 } }
I20260811 19:12:13.724658 17978 tablet_bootstrap.cc:492] T bbb8f840c6fa4dbda17bee1e22b6088a P 8b314dd688dd4682b73c26337f53a6cd: No bootstrap required, opened a new log
I20260811 19:12:13.725312 17978 ts_tablet_manager.cc:1397] T bbb8f840c6fa4dbda17bee1e22b6088a P 8b314dd688dd4682b73c26337f53a6cd: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260811 19:12:13.726754 17708 heartbeater.cc:503] Master 127.3.61.190:45797 was elected leader, sending a full tablet report...
I20260811 19:12:13.728025 17975 ts_tablet_manager.cc:1428] T bbb8f840c6fa4dbda17bee1e22b6088a P 7f82b1726e1d4ee7849c8692b83c0035: Time spent starting tablet: real 0.019s user 0.011s sys 0.009s
I20260811 19:12:13.729764 17977 tablet_bootstrap.cc:492] T bbb8f840c6fa4dbda17bee1e22b6088a P 2ea91336741c42ae9f80f61152e39ad9: No bootstrap required, opened a new log
I20260811 19:12:13.730078 17977 ts_tablet_manager.cc:1397] T bbb8f840c6fa4dbda17bee1e22b6088a P 2ea91336741c42ae9f80f61152e39ad9: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.007s
I20260811 19:12:13.735715 17978 raft_consensus.cc:348] T bbb8f840c6fa4dbda17bee1e22b6088a P 8b314dd688dd4682b73c26337f53a6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 } } peers { permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 } } peers { permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 } }
I20260811 19:12:13.736163 17978 raft_consensus.cc:374] T bbb8f840c6fa4dbda17bee1e22b6088a P 8b314dd688dd4682b73c26337f53a6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:12:13.736276 17978 raft_consensus.cc:729] T bbb8f840c6fa4dbda17bee1e22b6088a P 8b314dd688dd4682b73c26337f53a6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b314dd688dd4682b73c26337f53a6cd, State: Initialized, Role: FOLLOWER
I20260811 19:12:13.736892 17978 consensus_queue.cc:260] T bbb8f840c6fa4dbda17bee1e22b6088a P 8b314dd688dd4682b73c26337f53a6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 } } peers { permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 } } peers { permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 } }
I20260811 19:12:13.738878 17978 ts_tablet_manager.cc:1428] T bbb8f840c6fa4dbda17bee1e22b6088a P 8b314dd688dd4682b73c26337f53a6cd: Time spent starting tablet: real 0.013s user 0.010s sys 0.006s
I20260811 19:12:13.739156 17977 raft_consensus.cc:348] T bbb8f840c6fa4dbda17bee1e22b6088a P 2ea91336741c42ae9f80f61152e39ad9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 } } peers { permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 } } peers { permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 } }
I20260811 19:12:13.739595 17977 raft_consensus.cc:374] T bbb8f840c6fa4dbda17bee1e22b6088a P 2ea91336741c42ae9f80f61152e39ad9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:12:13.739696 17977 raft_consensus.cc:729] T bbb8f840c6fa4dbda17bee1e22b6088a P 2ea91336741c42ae9f80f61152e39ad9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ea91336741c42ae9f80f61152e39ad9, State: Initialized, Role: FOLLOWER
I20260811 19:12:13.740327 17977 consensus_queue.cc:260] T bbb8f840c6fa4dbda17bee1e22b6088a P 2ea91336741c42ae9f80f61152e39ad9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 } } peers { permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 } } peers { permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 } }
I20260811 19:12:13.741494 17936 heartbeater.cc:503] Master 127.3.61.190:45797 was elected leader, sending a full tablet report...
I20260811 19:12:13.742529 17977 ts_tablet_manager.cc:1428] T bbb8f840c6fa4dbda17bee1e22b6088a P 2ea91336741c42ae9f80f61152e39ad9: Time spent starting tablet: real 0.012s user 0.011s sys 0.001s
W20260811 19:12:13.784965 17937 tablet.cc:2369] T bbb8f840c6fa4dbda17bee1e22b6088a P 2ea91336741c42ae9f80f61152e39ad9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:12:13.787497 17983 raft_consensus.cc:482] T bbb8f840c6fa4dbda17bee1e22b6088a P 7f82b1726e1d4ee7849c8692b83c0035 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:12:13.787964 17983 raft_consensus.cc:504] T bbb8f840c6fa4dbda17bee1e22b6088a P 7f82b1726e1d4ee7849c8692b83c0035 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 } } peers { permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 } } peers { permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 } }
W20260811 19:12:13.788128 17479 tablet.cc:2369] T bbb8f840c6fa4dbda17bee1e22b6088a P 8b314dd688dd4682b73c26337f53a6cd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:12:13.790155 17983 leader_election.cc:288] T bbb8f840c6fa4dbda17bee1e22b6088a P 7f82b1726e1d4ee7849c8692b83c0035 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b314dd688dd4682b73c26337f53a6cd (127.3.61.129:45885), 2ea91336741c42ae9f80f61152e39ad9 (127.3.61.131:36703)
I20260811 19:12:13.800617 17987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:13.790662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.131:36703 (local address 127.3.61.130:42651)
0811 19:12:13.791877 (+ 1215us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:13.791900 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:13.792175 (+ 275us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:13.792625 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:13.792630 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:13.792693 (+ 63us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:12:13.793202 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:13.793220 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:13.794763 (+ 1543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:13.794768 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:13.797499 (+ 2731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:13.797522 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:13.799522 (+ 2000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:13.799530 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:13.799723 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:13.799768 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 19:12:13.799875 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 19:12:13.800142 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1070,"mutex_wait_us":237,"spinlock_wait_cycles":28160,"thread_start_us":528,"threads_started":1}
I20260811 19:12:13.800658 17974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:13.790711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:42651 (local address 127.3.61.131:36703)
0811 19:12:13.790963 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:13.790970 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:13.792076 (+ 1106us) server_negotiation.cc:407] Connection header received
0811 19:12:13.792348 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:13.792355 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:13.792459 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:13.792562 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:12:13.793425 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:13.794593 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:13.797850 (+ 3257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:13.799126 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:13.799356 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:13.799444 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:13.800036 (+ 592us) server_negotiation.cc:299] Negotiation successful
0811 19:12:13.800252 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260811 19:12:13.802134 17969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:13.790809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:59325 (local address 127.3.61.129:45885)
0811 19:12:13.791032 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:13.791039 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:13.792075 (+ 1036us) server_negotiation.cc:407] Connection header received
0811 19:12:13.792348 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:13.792355 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:13.792459 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:13.792562 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:12:13.793423 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:13.794595 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:13.799279 (+ 4684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:13.800788 (+ 1509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:13.801006 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:13.801085 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:13.801772 (+ 687us) server_negotiation.cc:299] Negotiation successful
0811 19:12:13.801915 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260811 19:12:13.802294 17991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:13.790662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:45885 (local address 127.3.61.130:59325)
0811 19:12:13.791877 (+ 1215us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:13.791900 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:13.792175 (+ 275us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:13.792625 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:13.792630 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:13.792815 (+ 185us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:12:13.793231 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:13.793241 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:13.794762 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:13.794768 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:13.797499 (+ 2731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:13.797522 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:13.801303 (+ 3781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:13.801309 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:13.801508 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:13.801538 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 19:12:13.801691 (+ 153us) client_negotiation.cc:240] Negotiation successful
0811 19:12:13.801866 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1022,"mutex_wait_us":141,"spinlock_wait_cycles":323072,"thread_start_us":587,"threads_started":1}
I20260811 19:12:13.802382 17856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbb8f840c6fa4dbda17bee1e22b6088a" candidate_uuid: "7f82b1726e1d4ee7849c8692b83c0035" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ea91336741c42ae9f80f61152e39ad9" is_pre_election: true
I20260811 19:12:13.803112 17856 raft_consensus.cc:2454] T bbb8f840c6fa4dbda17bee1e22b6088a P 2ea91336741c42ae9f80f61152e39ad9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f82b1726e1d4ee7849c8692b83c0035 in term 0.
I20260811 19:12:13.803418 17403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbb8f840c6fa4dbda17bee1e22b6088a" candidate_uuid: "7f82b1726e1d4ee7849c8692b83c0035" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b314dd688dd4682b73c26337f53a6cd" is_pre_election: true
I20260811 19:12:13.804008 17512 leader_election.cc:302] T bbb8f840c6fa4dbda17bee1e22b6088a P 7f82b1726e1d4ee7849c8692b83c0035 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ea91336741c42ae9f80f61152e39ad9, 7f82b1726e1d4ee7849c8692b83c0035; no voters:
I20260811 19:12:13.804095 17403 raft_consensus.cc:2454] T bbb8f840c6fa4dbda17bee1e22b6088a P 8b314dd688dd4682b73c26337f53a6cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f82b1726e1d4ee7849c8692b83c0035 in term 0.
I20260811 19:12:13.804476 17983 raft_consensus.cc:2781] T bbb8f840c6fa4dbda17bee1e22b6088a P 7f82b1726e1d4ee7849c8692b83c0035 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:12:13.804630 17983 raft_consensus.cc:482] T bbb8f840c6fa4dbda17bee1e22b6088a P 7f82b1726e1d4ee7849c8692b83c0035 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:12:13.804734 17983 raft_consensus.cc:3037] T bbb8f840c6fa4dbda17bee1e22b6088a P 7f82b1726e1d4ee7849c8692b83c0035 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:12:13.807843 17983 raft_consensus.cc:504] T bbb8f840c6fa4dbda17bee1e22b6088a P 7f82b1726e1d4ee7849c8692b83c0035 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 } } peers { permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 } } peers { permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 } }
I20260811 19:12:13.808612 17983 leader_election.cc:288] T bbb8f840c6fa4dbda17bee1e22b6088a P 7f82b1726e1d4ee7849c8692b83c0035 [CANDIDATE]: Term 1 election: Requested vote from peers 8b314dd688dd4682b73c26337f53a6cd (127.3.61.129:45885), 2ea91336741c42ae9f80f61152e39ad9 (127.3.61.131:36703)
I20260811 19:12:13.809188 17403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbb8f840c6fa4dbda17bee1e22b6088a" candidate_uuid: "7f82b1726e1d4ee7849c8692b83c0035" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b314dd688dd4682b73c26337f53a6cd"
I20260811 19:12:13.809255 17856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbb8f840c6fa4dbda17bee1e22b6088a" candidate_uuid: "7f82b1726e1d4ee7849c8692b83c0035" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ea91336741c42ae9f80f61152e39ad9"
I20260811 19:12:13.809422 17403 raft_consensus.cc:3037] T bbb8f840c6fa4dbda17bee1e22b6088a P 8b314dd688dd4682b73c26337f53a6cd [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:12:13.809492 17856 raft_consensus.cc:3037] T bbb8f840c6fa4dbda17bee1e22b6088a P 2ea91336741c42ae9f80f61152e39ad9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:12:13.812026 17856 raft_consensus.cc:2454] T bbb8f840c6fa4dbda17bee1e22b6088a P 2ea91336741c42ae9f80f61152e39ad9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f82b1726e1d4ee7849c8692b83c0035 in term 1.
I20260811 19:12:13.812336 17403 raft_consensus.cc:2454] T bbb8f840c6fa4dbda17bee1e22b6088a P 8b314dd688dd4682b73c26337f53a6cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f82b1726e1d4ee7849c8692b83c0035 in term 1.
I20260811 19:12:13.812561 17512 leader_election.cc:302] T bbb8f840c6fa4dbda17bee1e22b6088a P 7f82b1726e1d4ee7849c8692b83c0035 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ea91336741c42ae9f80f61152e39ad9, 7f82b1726e1d4ee7849c8692b83c0035; no voters:
I20260811 19:12:13.812951 17983 raft_consensus.cc:2781] T bbb8f840c6fa4dbda17bee1e22b6088a P 7f82b1726e1d4ee7849c8692b83c0035 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:12:13.813422 17983 raft_consensus.cc:686] T bbb8f840c6fa4dbda17bee1e22b6088a P 7f82b1726e1d4ee7849c8692b83c0035 [term 1 LEADER]: Becoming Leader. State: Replica: 7f82b1726e1d4ee7849c8692b83c0035, State: Running, Role: LEADER
I20260811 19:12:13.814026 17983 consensus_queue.cc:237] T bbb8f840c6fa4dbda17bee1e22b6088a P 7f82b1726e1d4ee7849c8692b83c0035 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 } } peers { permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 } } peers { permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 } }
I20260811 19:12:13.820024 17131 catalog_manager.cc:5495] T bbb8f840c6fa4dbda17bee1e22b6088a P 7f82b1726e1d4ee7849c8692b83c0035 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7f82b1726e1d4ee7849c8692b83c0035 (127.3.61.130). New cstate: current_term: 1 leader_uuid: "7f82b1726e1d4ee7849c8692b83c0035" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 } health_report { overall_health: UNKNOWN } } }
I20260811 19:12:13.845599 17967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:13.838775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59550 (local address 127.3.61.130:45219)
0811 19:12:13.839013 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:13.839020 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:13.839044 (+ 24us) server_negotiation.cc:407] Connection header received
0811 19:12:13.839340 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:13.839346 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:13.839448 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:13.839613 (+ 165us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:12:13.840176 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:13.841043 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:13.842838 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:13.843331 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:13.843466 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:13.844072 (+ 606us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:12:13.844742 (+ 670us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:12:13.844902 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:13.845188 (+ 286us) server_negotiation.cc:299] Negotiation successful
0811 19:12:13.845294 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 19:12:13.856037 17403 raft_consensus.cc:1261] T bbb8f840c6fa4dbda17bee1e22b6088a P 8b314dd688dd4682b73c26337f53a6cd [term 1 FOLLOWER]: Refusing update from remote peer 7f82b1726e1d4ee7849c8692b83c0035: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:12:13.856037 17856 raft_consensus.cc:1261] T bbb8f840c6fa4dbda17bee1e22b6088a P 2ea91336741c42ae9f80f61152e39ad9 [term 1 FOLLOWER]: Refusing update from remote peer 7f82b1726e1d4ee7849c8692b83c0035: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:12:13.857067 17992 consensus_queue.cc:1035] T bbb8f840c6fa4dbda17bee1e22b6088a P 7f82b1726e1d4ee7849c8692b83c0035 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:12:13.857434 17983 consensus_queue.cc:1035] T bbb8f840c6fa4dbda17bee1e22b6088a P 7f82b1726e1d4ee7849c8692b83c0035 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:12:13.869251 17995 mvcc.cc:205] Tried to move back new op lower bound from 7317403786663702528 to 7317403786506752000. Current Snapshot: MvccSnapshot[applied={T|T < 7317403786663702528}]
I20260811 19:12:13.895964 17135 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42444:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:12:13.896301 17135 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42444:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:12:13.900782 17135 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 21afe580e560414d97610b190fba81cd: Sending DeleteTablet for 3 replicas of tablet bbb8f840c6fa4dbda17bee1e22b6088a
I20260811 19:12:13.901975 17597 tablet_service.cc:1514] Processing DeleteTablet for tablet bbb8f840c6fa4dbda17bee1e22b6088a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:12:13 UTC) from {username='oryx', principal='oryx/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:59546
I20260811 19:12:13.903118 17375 tablet_service.cc:1514] Processing DeleteTablet for tablet bbb8f840c6fa4dbda17bee1e22b6088a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:12:13 UTC) from {username='oryx', principal='oryx/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:41170
I20260811 19:12:13.907792 17824 tablet_service.cc:1514] Processing DeleteTablet for tablet bbb8f840c6fa4dbda17bee1e22b6088a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:12:13 UTC) from {username='oryx', principal='oryx/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:54366
I20260811 19:12:13.909617 18012 tablet_replica.cc:330] T bbb8f840c6fa4dbda17bee1e22b6088a P 8b314dd688dd4682b73c26337f53a6cd: stopping tablet replica
I20260811 19:12:13.910168 17135 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42444:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 19:12:13.910874 17135 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:12:13.911916 18012 raft_consensus.cc:2229] T bbb8f840c6fa4dbda17bee1e22b6088a P 8b314dd688dd4682b73c26337f53a6cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:12:13.912532 18012 raft_consensus.cc:2258] T bbb8f840c6fa4dbda17bee1e22b6088a P 8b314dd688dd4682b73c26337f53a6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:12:13.918375 18012 ts_tablet_manager.cc:1905] T bbb8f840c6fa4dbda17bee1e22b6088a P 8b314dd688dd4682b73c26337f53a6cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:12:13.919489 18013 tablet_replica.cc:330] T bbb8f840c6fa4dbda17bee1e22b6088a P 2ea91336741c42ae9f80f61152e39ad9: stopping tablet replica
I20260811 19:12:13.920101 18011 tablet_replica.cc:330] T bbb8f840c6fa4dbda17bee1e22b6088a P 7f82b1726e1d4ee7849c8692b83c0035: stopping tablet replica
I20260811 19:12:13.920331 17597 tablet_service.cc:1467] Processing CreateTablet for tablet 134b00ed85444b109bf267009a10ce41 (DEFAULT_TABLE table=test-table [id=b49b142b52bd463e95a80cddb13d9418]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:12:13.920863 18011 raft_consensus.cc:2229] T bbb8f840c6fa4dbda17bee1e22b6088a P 7f82b1726e1d4ee7849c8692b83c0035 [term 1 LEADER]: Raft consensus shutting down.
I20260811 19:12:13.921245 17375 tablet_service.cc:1467] Processing CreateTablet for tablet 134b00ed85444b109bf267009a10ce41 (DEFAULT_TABLE table=test-table [id=b49b142b52bd463e95a80cddb13d9418]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:12:13.921387 17597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 134b00ed85444b109bf267009a10ce41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:12:13.922143 18013 raft_consensus.cc:2229] T bbb8f840c6fa4dbda17bee1e22b6088a P 2ea91336741c42ae9f80f61152e39ad9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:12:13.922343 17375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 134b00ed85444b109bf267009a10ce41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:12:13.922658 18013 raft_consensus.cc:2258] T bbb8f840c6fa4dbda17bee1e22b6088a P 2ea91336741c42ae9f80f61152e39ad9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:12:13.923035 17824 tablet_service.cc:1467] Processing CreateTablet for tablet 134b00ed85444b109bf267009a10ce41 (DEFAULT_TABLE table=test-table [id=b49b142b52bd463e95a80cddb13d9418]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:12:13.923496 17824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 134b00ed85444b109bf267009a10ce41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:12:13.925235 18013 ts_tablet_manager.cc:1905] T bbb8f840c6fa4dbda17bee1e22b6088a P 2ea91336741c42ae9f80f61152e39ad9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:12:13.928045 18012 ts_tablet_manager.cc:1918] T bbb8f840c6fa4dbda17bee1e22b6088a P 8b314dd688dd4682b73c26337f53a6cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:12:13.928223 18012 log.cc:1196] T bbb8f840c6fa4dbda17bee1e22b6088a P 8b314dd688dd4682b73c26337f53a6cd: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-0/wal/wals/bbb8f840c6fa4dbda17bee1e22b6088a
I20260811 19:12:13.928769 18012 ts_tablet_manager.cc:1939] T bbb8f840c6fa4dbda17bee1e22b6088a P 8b314dd688dd4682b73c26337f53a6cd: Deleting consensus metadata
I20260811 19:12:13.929452 17978 tablet_bootstrap.cc:492] T 134b00ed85444b109bf267009a10ce41 P 8b314dd688dd4682b73c26337f53a6cd: Bootstrap starting.
I20260811 19:12:13.931236 17978 tablet_bootstrap.cc:654] T 134b00ed85444b109bf267009a10ce41 P 8b314dd688dd4682b73c26337f53a6cd: Neither blocks nor log segments found. Creating new log.
I20260811 19:12:13.931385 17977 tablet_bootstrap.cc:492] T 134b00ed85444b109bf267009a10ce41 P 2ea91336741c42ae9f80f61152e39ad9: Bootstrap starting.
I20260811 19:12:13.932994 17977 tablet_bootstrap.cc:654] T 134b00ed85444b109bf267009a10ce41 P 2ea91336741c42ae9f80f61152e39ad9: Neither blocks nor log segments found. Creating new log.
I20260811 19:12:13.936280 18011 raft_consensus.cc:2258] T bbb8f840c6fa4dbda17bee1e22b6088a P 7f82b1726e1d4ee7849c8692b83c0035 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:12:13.937673 17975 tablet_bootstrap.cc:492] T 134b00ed85444b109bf267009a10ce41 P 7f82b1726e1d4ee7849c8692b83c0035: Bootstrap starting.
I20260811 19:12:13.939411 17975 tablet_bootstrap.cc:654] T 134b00ed85444b109bf267009a10ce41 P 7f82b1726e1d4ee7849c8692b83c0035: Neither blocks nor log segments found. Creating new log.
I20260811 19:12:13.945122 18011 ts_tablet_manager.cc:1905] T bbb8f840c6fa4dbda17bee1e22b6088a P 7f82b1726e1d4ee7849c8692b83c0035: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:12:13.947124 17978 tablet_bootstrap.cc:492] T 134b00ed85444b109bf267009a10ce41 P 8b314dd688dd4682b73c26337f53a6cd: No bootstrap required, opened a new log
I20260811 19:12:13.947346 17978 ts_tablet_manager.cc:1397] T 134b00ed85444b109bf267009a10ce41 P 8b314dd688dd4682b73c26337f53a6cd: Time spent bootstrapping tablet: real 0.018s user 0.004s sys 0.000s
I20260811 19:12:13.948314 17978 raft_consensus.cc:348] T 134b00ed85444b109bf267009a10ce41 P 8b314dd688dd4682b73c26337f53a6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 } } peers { permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 } } peers { permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 } }
I20260811 19:12:13.948606 17978 raft_consensus.cc:374] T 134b00ed85444b109bf267009a10ce41 P 8b314dd688dd4682b73c26337f53a6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:12:13.948696 17978 raft_consensus.cc:729] T 134b00ed85444b109bf267009a10ce41 P 8b314dd688dd4682b73c26337f53a6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b314dd688dd4682b73c26337f53a6cd, State: Initialized, Role: FOLLOWER
I20260811 19:12:13.949046 17978 consensus_queue.cc:260] T 134b00ed85444b109bf267009a10ce41 P 8b314dd688dd4682b73c26337f53a6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 } } peers { permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 } } peers { permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 } }
I20260811 19:12:13.950687 17978 ts_tablet_manager.cc:1428] T 134b00ed85444b109bf267009a10ce41 P 8b314dd688dd4682b73c26337f53a6cd: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 19:12:13.952577 17977 tablet_bootstrap.cc:492] T 134b00ed85444b109bf267009a10ce41 P 2ea91336741c42ae9f80f61152e39ad9: No bootstrap required, opened a new log
I20260811 19:12:13.952755 17977 ts_tablet_manager.cc:1397] T 134b00ed85444b109bf267009a10ce41 P 2ea91336741c42ae9f80f61152e39ad9: Time spent bootstrapping tablet: real 0.021s user 0.016s sys 0.004s
I20260811 19:12:13.953632 17977 raft_consensus.cc:348] T 134b00ed85444b109bf267009a10ce41 P 2ea91336741c42ae9f80f61152e39ad9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 } } peers { permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 } } peers { permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 } }
I20260811 19:12:13.953902 17977 raft_consensus.cc:374] T 134b00ed85444b109bf267009a10ce41 P 2ea91336741c42ae9f80f61152e39ad9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:12:13.953995 17977 raft_consensus.cc:729] T 134b00ed85444b109bf267009a10ce41 P 2ea91336741c42ae9f80f61152e39ad9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ea91336741c42ae9f80f61152e39ad9, State: Initialized, Role: FOLLOWER
I20260811 19:12:13.954296 17977 consensus_queue.cc:260] T 134b00ed85444b109bf267009a10ce41 P 2ea91336741c42ae9f80f61152e39ad9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 } } peers { permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 } } peers { permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 } }
I20260811 19:12:13.955000 17977 ts_tablet_manager.cc:1428] T 134b00ed85444b109bf267009a10ce41 P 2ea91336741c42ae9f80f61152e39ad9: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260811 19:12:13.957794 17109 catalog_manager.cc:4841] TS 8b314dd688dd4682b73c26337f53a6cd (127.3.61.129:45885): tablet bbb8f840c6fa4dbda17bee1e22b6088a (table test-table [id=44d4c5eb8ab644099b900dca4dbf1f00]) successfully deleted
I20260811 19:12:13.962069 18011 ts_tablet_manager.cc:1918] T bbb8f840c6fa4dbda17bee1e22b6088a P 7f82b1726e1d4ee7849c8692b83c0035: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:12:13.962278 18011 log.cc:1196] T bbb8f840c6fa4dbda17bee1e22b6088a P 7f82b1726e1d4ee7849c8692b83c0035: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-1/wal/wals/bbb8f840c6fa4dbda17bee1e22b6088a
I20260811 19:12:13.962719 18013 ts_tablet_manager.cc:1918] T bbb8f840c6fa4dbda17bee1e22b6088a P 2ea91336741c42ae9f80f61152e39ad9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:12:13.962884 18011 ts_tablet_manager.cc:1939] T bbb8f840c6fa4dbda17bee1e22b6088a P 7f82b1726e1d4ee7849c8692b83c0035: Deleting consensus metadata
I20260811 19:12:13.963001 18013 log.cc:1196] T bbb8f840c6fa4dbda17bee1e22b6088a P 2ea91336741c42ae9f80f61152e39ad9: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-2/wal/wals/bbb8f840c6fa4dbda17bee1e22b6088a
I20260811 19:12:13.963196 17975 tablet_bootstrap.cc:492] T 134b00ed85444b109bf267009a10ce41 P 7f82b1726e1d4ee7849c8692b83c0035: No bootstrap required, opened a new log
I20260811 19:12:13.963371 17975 ts_tablet_manager.cc:1397] T 134b00ed85444b109bf267009a10ce41 P 7f82b1726e1d4ee7849c8692b83c0035: Time spent bootstrapping tablet: real 0.026s user 0.009s sys 0.007s
I20260811 19:12:13.964393 17975 raft_consensus.cc:348] T 134b00ed85444b109bf267009a10ce41 P 7f82b1726e1d4ee7849c8692b83c0035 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 } } peers { permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 } } peers { permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 } }
I20260811 19:12:13.964648 17975 raft_consensus.cc:374] T 134b00ed85444b109bf267009a10ce41 P 7f82b1726e1d4ee7849c8692b83c0035 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:12:13.964753 17975 raft_consensus.cc:729] T 134b00ed85444b109bf267009a10ce41 P 7f82b1726e1d4ee7849c8692b83c0035 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f82b1726e1d4ee7849c8692b83c0035, State: Initialized, Role: FOLLOWER
I20260811 19:12:13.965140 17111 catalog_manager.cc:4841] TS 7f82b1726e1d4ee7849c8692b83c0035 (127.3.61.130:45219): tablet bbb8f840c6fa4dbda17bee1e22b6088a (table test-table [id=44d4c5eb8ab644099b900dca4dbf1f00]) successfully deleted
I20260811 19:12:13.965045 17975 consensus_queue.cc:260] T 134b00ed85444b109bf267009a10ce41 P 7f82b1726e1d4ee7849c8692b83c0035 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 } } peers { permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 } } peers { permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 } }
I20260811 19:12:13.965749 17975 ts_tablet_manager.cc:1428] T 134b00ed85444b109bf267009a10ce41 P 7f82b1726e1d4ee7849c8692b83c0035: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 19:12:13.966207 18013 ts_tablet_manager.cc:1939] T bbb8f840c6fa4dbda17bee1e22b6088a P 2ea91336741c42ae9f80f61152e39ad9: Deleting consensus metadata
I20260811 19:12:13.967831 17110 catalog_manager.cc:4841] TS 2ea91336741c42ae9f80f61152e39ad9 (127.3.61.131:36703): tablet bbb8f840c6fa4dbda17bee1e22b6088a (table test-table [id=44d4c5eb8ab644099b900dca4dbf1f00]) successfully deleted
I20260811 19:12:14.038913 17985 raft_consensus.cc:482] T 134b00ed85444b109bf267009a10ce41 P 2ea91336741c42ae9f80f61152e39ad9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:12:14.039177 17985 raft_consensus.cc:504] T 134b00ed85444b109bf267009a10ce41 P 2ea91336741c42ae9f80f61152e39ad9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 } } peers { permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 } } peers { permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 } }
I20260811 19:12:14.040777 17985 leader_election.cc:288] T 134b00ed85444b109bf267009a10ce41 P 2ea91336741c42ae9f80f61152e39ad9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b314dd688dd4682b73c26337f53a6cd (127.3.61.129:45885), 7f82b1726e1d4ee7849c8692b83c0035 (127.3.61.130:45219)
I20260811 19:12:14.049844 18022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:14.041129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.130:45219 (local address 127.3.61.131:60789)
0811 19:12:14.041832 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:14.041854 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:14.042019 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:14.042645 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:14.042649 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:14.042677 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:12:14.042884 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:14.042896 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:14.044640 (+ 1744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:14.044646 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:14.046947 (+ 2301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:14.046965 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:14.048677 (+ 1712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:14.048683 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:14.048991 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:14.049084 (+ 93us) client_negotiation.cc:769] Sending connection context
0811 19:12:14.049234 (+ 150us) client_negotiation.cc:240] Negotiation successful
0811 19:12:14.049398 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"mutex_wait_us":75,"thread_start_us":805,"threads_started":1}
I20260811 19:12:14.049841 17967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:14.041700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:60789 (local address 127.3.61.130:45219)
0811 19:12:14.041905 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:14.041912 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:14.041938 (+ 26us) server_negotiation.cc:407] Connection header received
0811 19:12:14.042241 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:14.042249 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:14.042333 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:14.042397 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:12:14.043085 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:14.044489 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:14.047177 (+ 2688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:14.048307 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:14.048473 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:14.048552 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:14.049288 (+ 736us) server_negotiation.cc:299] Negotiation successful
0811 19:12:14.049446 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 19:12:14.050009 17969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:14.041258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:44687 (local address 127.3.61.129:45885)
0811 19:12:14.041449 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:14.041455 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:14.042672 (+ 1217us) server_negotiation.cc:407] Connection header received
0811 19:12:14.042973 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:14.042979 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:14.043077 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:14.043159 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:12:14.043835 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:14.044569 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:14.047149 (+ 2580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:14.048468 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:14.048654 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:14.048731 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:14.049567 (+ 836us) server_negotiation.cc:299] Negotiation successful
0811 19:12:14.049748 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 19:12:14.050750 18023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:14.041439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:45885 (local address 127.3.61.131:44687)
0811 19:12:14.042545 (+ 1106us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:14.042571 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:14.042792 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:14.043190 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:14.043194 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:14.043218 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:12:14.043657 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:14.043672 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:14.044733 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:14.044737 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:14.046946 (+ 2209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:14.046965 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:14.048933 (+ 1968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:14.048939 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:14.049380 (+ 441us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:14.049411 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 19:12:14.050247 (+ 836us) client_negotiation.cc:240] Negotiation successful
0811 19:12:14.050413 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":982,"mutex_wait_us":64,"thread_start_us":843,"threads_started":1}
I20260811 19:12:14.051278 17626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "134b00ed85444b109bf267009a10ce41" candidate_uuid: "2ea91336741c42ae9f80f61152e39ad9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f82b1726e1d4ee7849c8692b83c0035" is_pre_election: true
I20260811 19:12:14.051481 17403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "134b00ed85444b109bf267009a10ce41" candidate_uuid: "2ea91336741c42ae9f80f61152e39ad9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b314dd688dd4682b73c26337f53a6cd" is_pre_election: true
I20260811 19:12:14.051734 17403 raft_consensus.cc:2454] T 134b00ed85444b109bf267009a10ce41 P 8b314dd688dd4682b73c26337f53a6cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ea91336741c42ae9f80f61152e39ad9 in term 0.
I20260811 19:12:14.051805 17626 raft_consensus.cc:2454] T 134b00ed85444b109bf267009a10ce41 P 7f82b1726e1d4ee7849c8692b83c0035 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ea91336741c42ae9f80f61152e39ad9 in term 0.
I20260811 19:12:14.052349 17735 leader_election.cc:302] T 134b00ed85444b109bf267009a10ce41 P 2ea91336741c42ae9f80f61152e39ad9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ea91336741c42ae9f80f61152e39ad9, 8b314dd688dd4682b73c26337f53a6cd; no voters:
I20260811 19:12:14.052791 17985 raft_consensus.cc:2781] T 134b00ed85444b109bf267009a10ce41 P 2ea91336741c42ae9f80f61152e39ad9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:12:14.052930 17985 raft_consensus.cc:482] T 134b00ed85444b109bf267009a10ce41 P 2ea91336741c42ae9f80f61152e39ad9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:12:14.053037 17985 raft_consensus.cc:3037] T 134b00ed85444b109bf267009a10ce41 P 2ea91336741c42ae9f80f61152e39ad9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:12:14.055336 17985 raft_consensus.cc:504] T 134b00ed85444b109bf267009a10ce41 P 2ea91336741c42ae9f80f61152e39ad9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 } } peers { permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 } } peers { permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 } }
I20260811 19:12:14.056028 17985 leader_election.cc:288] T 134b00ed85444b109bf267009a10ce41 P 2ea91336741c42ae9f80f61152e39ad9 [CANDIDATE]: Term 1 election: Requested vote from peers 8b314dd688dd4682b73c26337f53a6cd (127.3.61.129:45885), 7f82b1726e1d4ee7849c8692b83c0035 (127.3.61.130:45219)
I20260811 19:12:14.056620 17403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "134b00ed85444b109bf267009a10ce41" candidate_uuid: "2ea91336741c42ae9f80f61152e39ad9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b314dd688dd4682b73c26337f53a6cd"
I20260811 19:12:14.056874 17403 raft_consensus.cc:3037] T 134b00ed85444b109bf267009a10ce41 P 8b314dd688dd4682b73c26337f53a6cd [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:12:14.056844 17626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "134b00ed85444b109bf267009a10ce41" candidate_uuid: "2ea91336741c42ae9f80f61152e39ad9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f82b1726e1d4ee7849c8692b83c0035"
I20260811 19:12:14.057067 17626 raft_consensus.cc:3037] T 134b00ed85444b109bf267009a10ce41 P 7f82b1726e1d4ee7849c8692b83c0035 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:12:14.059974 17403 raft_consensus.cc:2454] T 134b00ed85444b109bf267009a10ce41 P 8b314dd688dd4682b73c26337f53a6cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ea91336741c42ae9f80f61152e39ad9 in term 1.
I20260811 19:12:14.060413 17626 raft_consensus.cc:2454] T 134b00ed85444b109bf267009a10ce41 P 7f82b1726e1d4ee7849c8692b83c0035 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ea91336741c42ae9f80f61152e39ad9 in term 1.
I20260811 19:12:14.060554 17735 leader_election.cc:302] T 134b00ed85444b109bf267009a10ce41 P 2ea91336741c42ae9f80f61152e39ad9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ea91336741c42ae9f80f61152e39ad9, 8b314dd688dd4682b73c26337f53a6cd; no voters:
I20260811 19:12:14.060914 17985 raft_consensus.cc:2781] T 134b00ed85444b109bf267009a10ce41 P 2ea91336741c42ae9f80f61152e39ad9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:12:14.061414 17985 raft_consensus.cc:686] T 134b00ed85444b109bf267009a10ce41 P 2ea91336741c42ae9f80f61152e39ad9 [term 1 LEADER]: Becoming Leader. State: Replica: 2ea91336741c42ae9f80f61152e39ad9, State: Running, Role: LEADER
I20260811 19:12:14.061872 17985 consensus_queue.cc:237] T 134b00ed85444b109bf267009a10ce41 P 2ea91336741c42ae9f80f61152e39ad9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 } } peers { permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 } } peers { permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 } }
I20260811 19:12:14.065726 17135 catalog_manager.cc:5495] T 134b00ed85444b109bf267009a10ce41 P 2ea91336741c42ae9f80f61152e39ad9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ea91336741c42ae9f80f61152e39ad9 (127.3.61.131). New cstate: current_term: 1 leader_uuid: "2ea91336741c42ae9f80f61152e39ad9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 } health_report { overall_health: HEALTHY } } }
Aug 11 19:12:14 dist-test-slave-t7g5 krb5kdc[17058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475531, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.61.190@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM
I20260811 19:12:14.112601 18035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:14.096870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:45797 (local address 127.0.0.1:42704)
0811 19:12:14.097578 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:14.097606 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:14.097780 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:14.098188 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:14.098191 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:14.098908 (+ 717us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:14.099199 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:14.099211 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:14.100345 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:14.100349 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:14.101392 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:14.101405 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:14.102138 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:14.102143 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:14.102278 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:14.103020 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:14.103045 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:14.105835 (+ 2790us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:12:14.109557 (+ 3722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:14.109571 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:14.109578 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:14.110073 (+ 495us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:14.110426 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:14.110439 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:14.110443 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:14.110594 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:14.111159 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:12:14.111168 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:12:14.111335 (+ 167us) client_negotiation.cc:769] Sending connection context
0811 19:12:14.111514 (+ 179us) client_negotiation.cc:240] Negotiation successful
0811 19:12:14.112090 (+ 576us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":398,"threads_started":1}
I20260811 19:12:14.112802 17751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:14.096870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42704 (local address 127.3.61.190:45797)
0811 19:12:14.097135 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:14.097142 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:14.097713 (+ 571us) server_negotiation.cc:407] Connection header received
0811 19:12:14.097896 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:14.097902 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:14.097990 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:14.098112 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:14.099364 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:14.100164 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:14.101522 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:14.101957 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:14.102067 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:14.106027 (+ 3960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:14.106033 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:14.106036 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:14.106066 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:14.109392 (+ 3326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:14.110187 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:14.110193 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:14.110195 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:14.110323 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:14.110705 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:14.110708 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:14.110710 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:14.111037 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:14.111167 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:14.111814 (+ 647us) server_negotiation.cc:299] Negotiation successful
0811 19:12:14.112349 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":180,"server-negotiator.queue_time_us":117}
I20260811 19:12:14.118638 17135 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:42704:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 19:12:14.120011 17135 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:12:14.127517 17375 tablet_service.cc:1467] Processing CreateTablet for tablet 19332da439114ece902802e55d308cf9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ed2564e3317b44818f4c07386e9f5e0a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 19:12:14.127660 17597 tablet_service.cc:1467] Processing CreateTablet for tablet 19332da439114ece902802e55d308cf9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ed2564e3317b44818f4c07386e9f5e0a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 19:12:14.128185 17375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19332da439114ece902802e55d308cf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:12:14.128286 17597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19332da439114ece902802e55d308cf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:12:14.131855 17824 tablet_service.cc:1467] Processing CreateTablet for tablet 19332da439114ece902802e55d308cf9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ed2564e3317b44818f4c07386e9f5e0a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 19:12:14.132647 17824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19332da439114ece902802e55d308cf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:12:14.136118 17975 tablet_bootstrap.cc:492] T 19332da439114ece902802e55d308cf9 P 7f82b1726e1d4ee7849c8692b83c0035: Bootstrap starting.
I20260811 19:12:14.137853 17975 tablet_bootstrap.cc:654] T 19332da439114ece902802e55d308cf9 P 7f82b1726e1d4ee7849c8692b83c0035: Neither blocks nor log segments found. Creating new log.
I20260811 19:12:14.139147 17977 tablet_bootstrap.cc:492] T 19332da439114ece902802e55d308cf9 P 2ea91336741c42ae9f80f61152e39ad9: Bootstrap starting.
W20260811 19:12:14.139849 17709 tablet.cc:2369] T 134b00ed85444b109bf267009a10ce41 P 7f82b1726e1d4ee7849c8692b83c0035: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:12:14.140779 17977 tablet_bootstrap.cc:654] T 19332da439114ece902802e55d308cf9 P 2ea91336741c42ae9f80f61152e39ad9: Neither blocks nor log segments found. Creating new log.
I20260811 19:12:14.142560 17975 tablet_bootstrap.cc:492] T 19332da439114ece902802e55d308cf9 P 7f82b1726e1d4ee7849c8692b83c0035: No bootstrap required, opened a new log
I20260811 19:12:14.142729 17975 ts_tablet_manager.cc:1397] T 19332da439114ece902802e55d308cf9 P 7f82b1726e1d4ee7849c8692b83c0035: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.003s
I20260811 19:12:14.143501 17975 raft_consensus.cc:348] T 19332da439114ece902802e55d308cf9 P 7f82b1726e1d4ee7849c8692b83c0035 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 } } peers { permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 } } peers { permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 } }
I20260811 19:12:14.143755 17975 raft_consensus.cc:374] T 19332da439114ece902802e55d308cf9 P 7f82b1726e1d4ee7849c8692b83c0035 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:12:14.143844 17975 raft_consensus.cc:729] T 19332da439114ece902802e55d308cf9 P 7f82b1726e1d4ee7849c8692b83c0035 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f82b1726e1d4ee7849c8692b83c0035, State: Initialized, Role: FOLLOWER
I20260811 19:12:14.144112 17975 consensus_queue.cc:260] T 19332da439114ece902802e55d308cf9 P 7f82b1726e1d4ee7849c8692b83c0035 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 } } peers { permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 } } peers { permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 } }
I20260811 19:12:14.144759 17975 ts_tablet_manager.cc:1428] T 19332da439114ece902802e55d308cf9 P 7f82b1726e1d4ee7849c8692b83c0035: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 19:12:14.144621 18005 tablet_replica.cc:439] T 19332da439114ece902802e55d308cf9 P 7f82b1726e1d4ee7849c8692b83c0035: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 } } peers { permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 } } peers { permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 } } }
I20260811 19:12:14.145021 18005 tablet_replica.cc:442] T 19332da439114ece902802e55d308cf9 P 7f82b1726e1d4ee7849c8692b83c0035: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 19:12:14.145428 17978 tablet_bootstrap.cc:492] T 19332da439114ece902802e55d308cf9 P 8b314dd688dd4682b73c26337f53a6cd: Bootstrap starting.
I20260811 19:12:14.147251 17978 tablet_bootstrap.cc:654] T 19332da439114ece902802e55d308cf9 P 8b314dd688dd4682b73c26337f53a6cd: Neither blocks nor log segments found. Creating new log.
I20260811 19:12:14.147431 17977 tablet_bootstrap.cc:492] T 19332da439114ece902802e55d308cf9 P 2ea91336741c42ae9f80f61152e39ad9: No bootstrap required, opened a new log
I20260811 19:12:14.147624 17977 ts_tablet_manager.cc:1397] T 19332da439114ece902802e55d308cf9 P 2ea91336741c42ae9f80f61152e39ad9: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.001s
I20260811 19:12:14.148588 17977 raft_consensus.cc:348] T 19332da439114ece902802e55d308cf9 P 2ea91336741c42ae9f80f61152e39ad9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 } } peers { permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 } } peers { permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 } }
I20260811 19:12:14.149075 17977 raft_consensus.cc:374] T 19332da439114ece902802e55d308cf9 P 2ea91336741c42ae9f80f61152e39ad9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:12:14.149334 17978 tablet_bootstrap.cc:492] T 19332da439114ece902802e55d308cf9 P 8b314dd688dd4682b73c26337f53a6cd: No bootstrap required, opened a new log
I20260811 19:12:14.149271 17977 raft_consensus.cc:729] T 19332da439114ece902802e55d308cf9 P 2ea91336741c42ae9f80f61152e39ad9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ea91336741c42ae9f80f61152e39ad9, State: Initialized, Role: FOLLOWER
I20260811 19:12:14.149502 17978 ts_tablet_manager.cc:1397] T 19332da439114ece902802e55d308cf9 P 8b314dd688dd4682b73c26337f53a6cd: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260811 19:12:14.149768 17977 consensus_queue.cc:260] T 19332da439114ece902802e55d308cf9 P 2ea91336741c42ae9f80f61152e39ad9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 } } peers { permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 } } peers { permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 } }
I20260811 19:12:14.150640 17977 ts_tablet_manager.cc:1428] T 19332da439114ece902802e55d308cf9 P 2ea91336741c42ae9f80f61152e39ad9: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 19:12:14.150481 17985 tablet_replica.cc:439] T 19332da439114ece902802e55d308cf9 P 2ea91336741c42ae9f80f61152e39ad9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 } } peers { permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 } } peers { permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 } } }
I20260811 19:12:14.150874 17985 tablet_replica.cc:442] T 19332da439114ece902802e55d308cf9 P 2ea91336741c42ae9f80f61152e39ad9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 19:12:14.150209 17978 raft_consensus.cc:348] T 19332da439114ece902802e55d308cf9 P 8b314dd688dd4682b73c26337f53a6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 } } peers { permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 } } peers { permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 } }
I20260811 19:12:14.153185 17978 raft_consensus.cc:374] T 19332da439114ece902802e55d308cf9 P 8b314dd688dd4682b73c26337f53a6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:12:14.153292 17978 raft_consensus.cc:729] T 19332da439114ece902802e55d308cf9 P 8b314dd688dd4682b73c26337f53a6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b314dd688dd4682b73c26337f53a6cd, State: Initialized, Role: FOLLOWER
I20260811 19:12:14.154572 17978 consensus_queue.cc:260] T 19332da439114ece902802e55d308cf9 P 8b314dd688dd4682b73c26337f53a6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 } } peers { permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 } } peers { permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 } }
I20260811 19:12:14.155258 17978 ts_tablet_manager.cc:1428] T 19332da439114ece902802e55d308cf9 P 8b314dd688dd4682b73c26337f53a6cd: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260811 19:12:14.155159 17984 tablet_replica.cc:439] T 19332da439114ece902802e55d308cf9 P 8b314dd688dd4682b73c26337f53a6cd: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 } } peers { permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 } } peers { permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 } } }
I20260811 19:12:14.155603 17984 tablet_replica.cc:442] T 19332da439114ece902802e55d308cf9 P 8b314dd688dd4682b73c26337f53a6cd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 19:12:14.191327 17985 raft_consensus.cc:482] T 19332da439114ece902802e55d308cf9 P 2ea91336741c42ae9f80f61152e39ad9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:12:14.191743 17985 raft_consensus.cc:504] T 19332da439114ece902802e55d308cf9 P 2ea91336741c42ae9f80f61152e39ad9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 } } peers { permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 } } peers { permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 } }
I20260811 19:12:14.192721 17985 leader_election.cc:288] T 19332da439114ece902802e55d308cf9 P 2ea91336741c42ae9f80f61152e39ad9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b314dd688dd4682b73c26337f53a6cd (127.3.61.129:45885), 7f82b1726e1d4ee7849c8692b83c0035 (127.3.61.130:45219)
I20260811 19:12:14.193921 17403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19332da439114ece902802e55d308cf9" candidate_uuid: "2ea91336741c42ae9f80f61152e39ad9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b314dd688dd4682b73c26337f53a6cd" is_pre_election: true
I20260811 19:12:14.193853 17626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19332da439114ece902802e55d308cf9" candidate_uuid: "2ea91336741c42ae9f80f61152e39ad9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f82b1726e1d4ee7849c8692b83c0035" is_pre_election: true
I20260811 19:12:14.194218 17403 raft_consensus.cc:2454] T 19332da439114ece902802e55d308cf9 P 8b314dd688dd4682b73c26337f53a6cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ea91336741c42ae9f80f61152e39ad9 in term 0.
I20260811 19:12:14.194224 17626 raft_consensus.cc:2454] T 19332da439114ece902802e55d308cf9 P 7f82b1726e1d4ee7849c8692b83c0035 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ea91336741c42ae9f80f61152e39ad9 in term 0.
I20260811 19:12:14.194784 17735 leader_election.cc:302] T 19332da439114ece902802e55d308cf9 P 2ea91336741c42ae9f80f61152e39ad9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ea91336741c42ae9f80f61152e39ad9, 8b314dd688dd4682b73c26337f53a6cd; no voters:
I20260811 19:12:14.195549 17985 raft_consensus.cc:2781] T 19332da439114ece902802e55d308cf9 P 2ea91336741c42ae9f80f61152e39ad9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:12:14.195648 17985 raft_consensus.cc:482] T 19332da439114ece902802e55d308cf9 P 2ea91336741c42ae9f80f61152e39ad9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:12:14.195763 17985 raft_consensus.cc:3037] T 19332da439114ece902802e55d308cf9 P 2ea91336741c42ae9f80f61152e39ad9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:12:14.198730 17985 raft_consensus.cc:504] T 19332da439114ece902802e55d308cf9 P 2ea91336741c42ae9f80f61152e39ad9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 } } peers { permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 } } peers { permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 } }
I20260811 19:12:14.199441 17985 leader_election.cc:288] T 19332da439114ece902802e55d308cf9 P 2ea91336741c42ae9f80f61152e39ad9 [CANDIDATE]: Term 1 election: Requested vote from peers 8b314dd688dd4682b73c26337f53a6cd (127.3.61.129:45885), 7f82b1726e1d4ee7849c8692b83c0035 (127.3.61.130:45219)
I20260811 19:12:14.199940 17403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19332da439114ece902802e55d308cf9" candidate_uuid: "2ea91336741c42ae9f80f61152e39ad9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b314dd688dd4682b73c26337f53a6cd"
I20260811 19:12:14.200107 17626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19332da439114ece902802e55d308cf9" candidate_uuid: "2ea91336741c42ae9f80f61152e39ad9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f82b1726e1d4ee7849c8692b83c0035"
I20260811 19:12:14.200232 17403 raft_consensus.cc:3037] T 19332da439114ece902802e55d308cf9 P 8b314dd688dd4682b73c26337f53a6cd [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:12:14.200372 17626 raft_consensus.cc:3037] T 19332da439114ece902802e55d308cf9 P 7f82b1726e1d4ee7849c8692b83c0035 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:12:14.203248 17403 raft_consensus.cc:2454] T 19332da439114ece902802e55d308cf9 P 8b314dd688dd4682b73c26337f53a6cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ea91336741c42ae9f80f61152e39ad9 in term 1.
I20260811 19:12:14.203531 17626 raft_consensus.cc:2454] T 19332da439114ece902802e55d308cf9 P 7f82b1726e1d4ee7849c8692b83c0035 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ea91336741c42ae9f80f61152e39ad9 in term 1.
I20260811 19:12:14.203807 17735 leader_election.cc:302] T 19332da439114ece902802e55d308cf9 P 2ea91336741c42ae9f80f61152e39ad9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ea91336741c42ae9f80f61152e39ad9, 8b314dd688dd4682b73c26337f53a6cd; no voters:
I20260811 19:12:14.204101 17985 raft_consensus.cc:2781] T 19332da439114ece902802e55d308cf9 P 2ea91336741c42ae9f80f61152e39ad9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:12:14.204468 17985 raft_consensus.cc:686] T 19332da439114ece902802e55d308cf9 P 2ea91336741c42ae9f80f61152e39ad9 [term 1 LEADER]: Becoming Leader. State: Replica: 2ea91336741c42ae9f80f61152e39ad9, State: Running, Role: LEADER
I20260811 19:12:14.204747 17985 consensus_queue.cc:237] T 19332da439114ece902802e55d308cf9 P 2ea91336741c42ae9f80f61152e39ad9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 } } peers { permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 } } peers { permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 } }
I20260811 19:12:14.206481 18024 tablet_replica.cc:439] T 19332da439114ece902802e55d308cf9 P 2ea91336741c42ae9f80f61152e39ad9: TxnStatusTablet state changed. Reason: New leader 2ea91336741c42ae9f80f61152e39ad9. Latest consensus state: current_term: 1 leader_uuid: "2ea91336741c42ae9f80f61152e39ad9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 } } peers { permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 } } peers { permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 } } }
I20260811 19:12:14.206898 18024 tablet_replica.cc:442] T 19332da439114ece902802e55d308cf9 P 2ea91336741c42ae9f80f61152e39ad9: This TxnStatusTablet replica's current role is: LEADER
I20260811 19:12:14.207626 18044 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 19:12:14.207849 18044 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 19:12:14.208359 17135 catalog_manager.cc:5495] T 19332da439114ece902802e55d308cf9 P 2ea91336741c42ae9f80f61152e39ad9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ea91336741c42ae9f80f61152e39ad9 (127.3.61.131). New cstate: current_term: 1 leader_uuid: "2ea91336741c42ae9f80f61152e39ad9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 } health_report { overall_health: HEALTHY } } }
I20260811 19:12:14.244306 18035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:14.237933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.131:36703 (local address 127.0.0.1:54370)
0811 19:12:14.238324 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:14.238344 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:14.238521 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:14.239026 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:14.239030 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:14.239042 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 19:12:14.239227 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:14.239236 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:14.240343 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:14.240349 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:14.241565 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:14.241581 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:14.242401 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:14.242407 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:14.242576 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:14.242644 (+ 68us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 19:12:14.243713 (+ 1069us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 19:12:14.243736 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 19:12:14.243824 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 19:12:14.244006 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191}
I20260811 19:12:14.244307 17974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:14.238044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54370 (local address 127.3.61.131:36703)
0811 19:12:14.238288 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:14.238294 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:14.238434 (+ 140us) server_negotiation.cc:407] Connection header received
0811 19:12:14.238732 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:14.238736 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:14.238847 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:14.238977 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:12:14.239354 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:14.240212 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:14.241771 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:14.242238 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:14.242352 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:14.242845 (+ 493us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:12:14.243579 (+ 734us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:12:14.243732 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:14.243856 (+ 124us) server_negotiation.cc:299] Negotiation successful
0811 19:12:14.243948 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260811 19:12:14.250558 17626 raft_consensus.cc:1261] T 19332da439114ece902802e55d308cf9 P 7f82b1726e1d4ee7849c8692b83c0035 [term 1 FOLLOWER]: Refusing update from remote peer 2ea91336741c42ae9f80f61152e39ad9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:12:14.251401 17403 raft_consensus.cc:1261] T 19332da439114ece902802e55d308cf9 P 8b314dd688dd4682b73c26337f53a6cd [term 1 FOLLOWER]: Refusing update from remote peer 2ea91336741c42ae9f80f61152e39ad9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:12:14.251834 17985 consensus_queue.cc:1035] T 19332da439114ece902802e55d308cf9 P 2ea91336741c42ae9f80f61152e39ad9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:12:14.252934 17985 consensus_queue.cc:1035] T 19332da439114ece902802e55d308cf9 P 2ea91336741c42ae9f80f61152e39ad9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:12:14.257551 17984 tablet_replica.cc:439] T 19332da439114ece902802e55d308cf9 P 8b314dd688dd4682b73c26337f53a6cd: TxnStatusTablet state changed. Reason: New leader 2ea91336741c42ae9f80f61152e39ad9. Latest consensus state: current_term: 1 leader_uuid: "2ea91336741c42ae9f80f61152e39ad9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 } } peers { permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 } } peers { permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 } } }
I20260811 19:12:14.258133 17984 tablet_replica.cc:442] T 19332da439114ece902802e55d308cf9 P 8b314dd688dd4682b73c26337f53a6cd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 19:12:14.262804 17985 tablet_replica.cc:439] T 19332da439114ece902802e55d308cf9 P 2ea91336741c42ae9f80f61152e39ad9: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2ea91336741c42ae9f80f61152e39ad9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 } } peers { permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 } } peers { permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 } } }
I20260811 19:12:14.263200 17985 tablet_replica.cc:442] T 19332da439114ece902802e55d308cf9 P 2ea91336741c42ae9f80f61152e39ad9: This TxnStatusTablet replica's current role is: LEADER
I20260811 19:12:14.266268 17998 mvcc.cc:205] Tried to move back new op lower bound from 7317403788278525952 to 7317403788105056256. Current Snapshot: MvccSnapshot[applied={T|T < 7317403788278525952}]
I20260811 19:12:14.266708 17984 tablet_replica.cc:439] T 19332da439114ece902802e55d308cf9 P 8b314dd688dd4682b73c26337f53a6cd: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2ea91336741c42ae9f80f61152e39ad9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 } } peers { permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 } } peers { permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 } } }
I20260811 19:12:14.267004 17984 tablet_replica.cc:442] T 19332da439114ece902802e55d308cf9 P 8b314dd688dd4682b73c26337f53a6cd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 19:12:14.274511 18005 tablet_replica.cc:439] T 19332da439114ece902802e55d308cf9 P 7f82b1726e1d4ee7849c8692b83c0035: TxnStatusTablet state changed. Reason: New leader 2ea91336741c42ae9f80f61152e39ad9. Latest consensus state: current_term: 1 leader_uuid: "2ea91336741c42ae9f80f61152e39ad9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 } } peers { permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 } } peers { permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 } } }
I20260811 19:12:14.274838 18005 tablet_replica.cc:442] T 19332da439114ece902802e55d308cf9 P 7f82b1726e1d4ee7849c8692b83c0035: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 19:12:14.279656 17974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:14.272249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54380 (local address 127.3.61.131:36703)
0811 19:12:14.272423 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:14.272428 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:14.272447 (+ 19us) server_negotiation.cc:407] Connection header received
0811 19:12:14.272655 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:14.272660 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:14.272765 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:14.272865 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:12:14.273812 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:14.274865 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:14.276926 (+ 2061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:14.277552 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:14.277702 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:14.278405 (+ 703us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:12:14.278544 (+ 139us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:12:14.278604 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:14.279211 (+ 607us) server_negotiation.cc:299] Negotiation successful
0811 19:12:14.279299 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 19:12:14.282847 18024 tablet_replica.cc:439] T 19332da439114ece902802e55d308cf9 P 2ea91336741c42ae9f80f61152e39ad9: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2ea91336741c42ae9f80f61152e39ad9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 } } peers { permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 } } peers { permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 } } }
I20260811 19:12:14.283099 18024 tablet_replica.cc:442] T 19332da439114ece902802e55d308cf9 P 2ea91336741c42ae9f80f61152e39ad9: This TxnStatusTablet replica's current role is: LEADER
I20260811 19:12:14.285065 18005 tablet_replica.cc:439] T 19332da439114ece902802e55d308cf9 P 7f82b1726e1d4ee7849c8692b83c0035: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2ea91336741c42ae9f80f61152e39ad9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 } } peers { permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 } } peers { permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 } } }
I20260811 19:12:14.285276 18005 tablet_replica.cc:442] T 19332da439114ece902802e55d308cf9 P 7f82b1726e1d4ee7849c8692b83c0035: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 19:12:14.336179 17974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:14.327992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:44597 (local address 127.3.61.131:36703)
0811 19:12:14.328422 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:14.328429 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:14.328982 (+ 553us) server_negotiation.cc:407] Connection header received
0811 19:12:14.329234 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:14.329240 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:14.329323 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:14.329433 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:12:14.330170 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:14.331387 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:14.333720 (+ 2333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:14.334442 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:14.334583 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:14.335183 (+ 600us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:12:14.335381 (+ 198us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:12:14.335493 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:14.335661 (+ 168us) server_negotiation.cc:299] Negotiation successful
0811 19:12:14.335758 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":7,"server-negotiator.queue_time_us":283}
I20260811 19:12:14.336277 18054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:14.327970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.131:36703 (local address 127.3.61.131:44597)
0811 19:12:14.328851 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:14.328873 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:14.329060 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:14.329474 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:14.329479 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:14.329492 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 19:12:14.329993 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:14.330008 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:14.331550 (+ 1542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:14.331559 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:14.333508 (+ 1949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:14.333528 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:14.334677 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:14.334685 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:14.334884 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:14.334976 (+ 92us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 19:12:14.335516 (+ 540us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 19:12:14.335530 (+ 14us) client_negotiation.cc:769] Sending connection context
0811 19:12:14.335634 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 19:12:14.335791 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":645,"mutex_wait_us":46,"thread_start_us":441,"threads_started":1}
I20260811 19:12:14.350461 17403 raft_consensus.cc:1261] T 134b00ed85444b109bf267009a10ce41 P 8b314dd688dd4682b73c26337f53a6cd [term 1 FOLLOWER]: Refusing update from remote peer 2ea91336741c42ae9f80f61152e39ad9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:12:14.351140 17985 consensus_queue.cc:1035] T 134b00ed85444b109bf267009a10ce41 P 2ea91336741c42ae9f80f61152e39ad9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:12:14.360530 17626 raft_consensus.cc:1261] T 134b00ed85444b109bf267009a10ce41 P 7f82b1726e1d4ee7849c8692b83c0035 [term 1 FOLLOWER]: Refusing update from remote peer 2ea91336741c42ae9f80f61152e39ad9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:12:14.364645 17997 mvcc.cc:205] Tried to move back new op lower bound from 7317403788693237760 to 7317403787521925120. Current Snapshot: MvccSnapshot[applied={T|T < 7317403788693237760}]
I20260811 19:12:14.366014 17985 consensus_queue.cc:1035] T 134b00ed85444b109bf267009a10ce41 P 2ea91336741c42ae9f80f61152e39ad9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:12:14.453338 17131 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42444:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:12:14.453631 17131 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42444:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:12:14.457345 17131 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 21afe580e560414d97610b190fba81cd: Sending DeleteTablet for 3 replicas of tablet 134b00ed85444b109bf267009a10ce41
I20260811 19:12:14.458312 17375 tablet_service.cc:1514] Processing DeleteTablet for tablet 134b00ed85444b109bf267009a10ce41 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:12:14 UTC) from {username='oryx', principal='oryx/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:41170
I20260811 19:12:14.458629 18012 tablet_replica.cc:330] T 134b00ed85444b109bf267009a10ce41 P 8b314dd688dd4682b73c26337f53a6cd: stopping tablet replica
I20260811 19:12:14.458619 17597 tablet_service.cc:1514] Processing DeleteTablet for tablet 134b00ed85444b109bf267009a10ce41 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:12:14 UTC) from {username='oryx', principal='oryx/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:59546
I20260811 19:12:14.458837 18012 raft_consensus.cc:2229] T 134b00ed85444b109bf267009a10ce41 P 8b314dd688dd4682b73c26337f53a6cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:12:14.458918 18011 tablet_replica.cc:330] T 134b00ed85444b109bf267009a10ce41 P 7f82b1726e1d4ee7849c8692b83c0035: stopping tablet replica
I20260811 19:12:14.459050 18012 raft_consensus.cc:2258] T 134b00ed85444b109bf267009a10ce41 P 8b314dd688dd4682b73c26337f53a6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:12:14.459121 18011 raft_consensus.cc:2229] T 134b00ed85444b109bf267009a10ce41 P 7f82b1726e1d4ee7849c8692b83c0035 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:12:14.459312 18011 raft_consensus.cc:2258] T 134b00ed85444b109bf267009a10ce41 P 7f82b1726e1d4ee7849c8692b83c0035 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:12:14.460021 18012 ts_tablet_manager.cc:1905] T 134b00ed85444b109bf267009a10ce41 P 8b314dd688dd4682b73c26337f53a6cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:12:14.460269 17824 tablet_service.cc:1514] Processing DeleteTablet for tablet 134b00ed85444b109bf267009a10ce41 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:12:14 UTC) from {username='oryx', principal='oryx/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:54366
I20260811 19:12:14.460590 18013 tablet_replica.cc:330] T 134b00ed85444b109bf267009a10ce41 P 2ea91336741c42ae9f80f61152e39ad9: stopping tablet replica
I20260811 19:12:14.460795 18013 raft_consensus.cc:2229] T 134b00ed85444b109bf267009a10ce41 P 2ea91336741c42ae9f80f61152e39ad9 [term 1 LEADER]: Raft consensus shutting down.
I20260811 19:12:14.461333 18013 raft_consensus.cc:2258] T 134b00ed85444b109bf267009a10ce41 P 2ea91336741c42ae9f80f61152e39ad9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:12:14.463234 18013 ts_tablet_manager.cc:1905] T 134b00ed85444b109bf267009a10ce41 P 2ea91336741c42ae9f80f61152e39ad9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:12:14.465238 18011 ts_tablet_manager.cc:1905] T 134b00ed85444b109bf267009a10ce41 P 7f82b1726e1d4ee7849c8692b83c0035: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:12:14.466802 18012 ts_tablet_manager.cc:1918] T 134b00ed85444b109bf267009a10ce41 P 8b314dd688dd4682b73c26337f53a6cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 19:12:14.466970 18012 log.cc:1196] T 134b00ed85444b109bf267009a10ce41 P 8b314dd688dd4682b73c26337f53a6cd: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-0/wal/wals/134b00ed85444b109bf267009a10ce41
I20260811 19:12:14.467394 18012 ts_tablet_manager.cc:1939] T 134b00ed85444b109bf267009a10ce41 P 8b314dd688dd4682b73c26337f53a6cd: Deleting consensus metadata
I20260811 19:12:14.468532 17109 catalog_manager.cc:4841] TS 8b314dd688dd4682b73c26337f53a6cd (127.3.61.129:45885): tablet 134b00ed85444b109bf267009a10ce41 (table test-table [id=b49b142b52bd463e95a80cddb13d9418]) successfully deleted
I20260811 19:12:14.471103 18013 ts_tablet_manager.cc:1918] T 134b00ed85444b109bf267009a10ce41 P 2ea91336741c42ae9f80f61152e39ad9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 19:12:14.471354 18013 log.cc:1196] T 134b00ed85444b109bf267009a10ce41 P 2ea91336741c42ae9f80f61152e39ad9: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-2/wal/wals/134b00ed85444b109bf267009a10ce41
I20260811 19:12:14.471854 18013 ts_tablet_manager.cc:1939] T 134b00ed85444b109bf267009a10ce41 P 2ea91336741c42ae9f80f61152e39ad9: Deleting consensus metadata
I20260811 19:12:14.473017 18011 ts_tablet_manager.cc:1918] T 134b00ed85444b109bf267009a10ce41 P 7f82b1726e1d4ee7849c8692b83c0035: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 19:12:14.473152 18011 log.cc:1196] T 134b00ed85444b109bf267009a10ce41 P 7f82b1726e1d4ee7849c8692b83c0035: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-1/wal/wals/134b00ed85444b109bf267009a10ce41
I20260811 19:12:14.473481 18011 ts_tablet_manager.cc:1939] T 134b00ed85444b109bf267009a10ce41 P 7f82b1726e1d4ee7849c8692b83c0035: Deleting consensus metadata
I20260811 19:12:14.474645 17111 catalog_manager.cc:4841] TS 7f82b1726e1d4ee7849c8692b83c0035 (127.3.61.130:45219): tablet 134b00ed85444b109bf267009a10ce41 (table test-table [id=b49b142b52bd463e95a80cddb13d9418]) successfully deleted
I20260811 19:12:14.474647 17110 catalog_manager.cc:4841] TS 2ea91336741c42ae9f80f61152e39ad9 (127.3.61.131:36703): tablet 134b00ed85444b109bf267009a10ce41 (table test-table [id=b49b142b52bd463e95a80cddb13d9418]) successfully deleted
Aug 11 19:12:14 dist-test-slave-t7g5 krb5kdc[17058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475534, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 19:12:14 dist-test-slave-t7g5 krb5kdc[17058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:12:14 dist-test-slave-t7g5 krb5kdc[17058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 19:12:14.541132 17751 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:12:14.526445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42714 (local address 127.3.61.190:45797)
0811 19:12:14.526950 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:14.526961 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:14.527695 (+ 734us) server_negotiation.cc:407] Connection header received
0811 19:12:14.527929 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:14.527934 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:14.528043 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:14.528138 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:14.529718 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:14.530991 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:14.533437 (+ 2446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:14.534262 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:14.534380 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:14.540879 (+ 6499us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42714: BlockingRecv error: recv got EOF from 127.0.0.1:42714 (error 108)
Metrics: {"server-negotiator.queue_time_us":287}
Aug 11 19:12:14 dist-test-slave-t7g5 krb5kdc[17058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475534, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM
I20260811 19:12:14.563867 17751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:14.544993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42728 (local address 127.3.61.190:45797)
0811 19:12:14.545193 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:14.545200 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:14.545739 (+ 539us) server_negotiation.cc:407] Connection header received
0811 19:12:14.545954 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:14.545960 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:14.546062 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:14.546166 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:14.547900 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:14.549066 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:14.550984 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:14.551620 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:14.551756 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:14.556785 (+ 5029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:14.556793 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:14.556798 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:14.556850 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:14.560348 (+ 3498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:14.561303 (+ 955us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:14.561309 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:14.561322 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:14.561480 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:14.561995 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:14.562001 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:14.562003 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:14.562445 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:14.562723 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:14.563241 (+ 518us) server_negotiation.cc:299] Negotiation successful
0811 19:12:14.563493 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 19:12:14.568253 17131 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42728:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 19:12:14.569044 17131 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:12:14.576786 17375 tablet_service.cc:1467] Processing CreateTablet for tablet 7716f27ea089411aa4aa4dd223ae7d69 (DEFAULT_TABLE table=test-table [id=ca3e1b226c8a4f10873c484162816632]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:12:14.577260 17375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7716f27ea089411aa4aa4dd223ae7d69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:12:14.577250 17597 tablet_service.cc:1467] Processing CreateTablet for tablet 7716f27ea089411aa4aa4dd223ae7d69 (DEFAULT_TABLE table=test-table [id=ca3e1b226c8a4f10873c484162816632]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:12:14.577729 17597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7716f27ea089411aa4aa4dd223ae7d69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:12:14.582176 17978 tablet_bootstrap.cc:492] T 7716f27ea089411aa4aa4dd223ae7d69 P 8b314dd688dd4682b73c26337f53a6cd: Bootstrap starting.
I20260811 19:12:14.582095 17824 tablet_service.cc:1467] Processing CreateTablet for tablet 7716f27ea089411aa4aa4dd223ae7d69 (DEFAULT_TABLE table=test-table [id=ca3e1b226c8a4f10873c484162816632]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:12:14.582546 17824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7716f27ea089411aa4aa4dd223ae7d69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:12:14.583889 17978 tablet_bootstrap.cc:654] T 7716f27ea089411aa4aa4dd223ae7d69 P 8b314dd688dd4682b73c26337f53a6cd: Neither blocks nor log segments found. Creating new log.
I20260811 19:12:14.585956 17975 tablet_bootstrap.cc:492] T 7716f27ea089411aa4aa4dd223ae7d69 P 7f82b1726e1d4ee7849c8692b83c0035: Bootstrap starting.
I20260811 19:12:14.587764 17975 tablet_bootstrap.cc:654] T 7716f27ea089411aa4aa4dd223ae7d69 P 7f82b1726e1d4ee7849c8692b83c0035: Neither blocks nor log segments found. Creating new log.
I20260811 19:12:14.589316 17977 tablet_bootstrap.cc:492] T 7716f27ea089411aa4aa4dd223ae7d69 P 2ea91336741c42ae9f80f61152e39ad9: Bootstrap starting.
I20260811 19:12:14.591327 17977 tablet_bootstrap.cc:654] T 7716f27ea089411aa4aa4dd223ae7d69 P 2ea91336741c42ae9f80f61152e39ad9: Neither blocks nor log segments found. Creating new log.
I20260811 19:12:14.597034 17978 tablet_bootstrap.cc:492] T 7716f27ea089411aa4aa4dd223ae7d69 P 8b314dd688dd4682b73c26337f53a6cd: No bootstrap required, opened a new log
I20260811 19:12:14.597227 17978 ts_tablet_manager.cc:1397] T 7716f27ea089411aa4aa4dd223ae7d69 P 8b314dd688dd4682b73c26337f53a6cd: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.009s
I20260811 19:12:14.598186 17978 raft_consensus.cc:348] T 7716f27ea089411aa4aa4dd223ae7d69 P 8b314dd688dd4682b73c26337f53a6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 } } peers { permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 } } peers { permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 } }
I20260811 19:12:14.598667 17978 raft_consensus.cc:374] T 7716f27ea089411aa4aa4dd223ae7d69 P 8b314dd688dd4682b73c26337f53a6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:12:14.598798 17978 raft_consensus.cc:729] T 7716f27ea089411aa4aa4dd223ae7d69 P 8b314dd688dd4682b73c26337f53a6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b314dd688dd4682b73c26337f53a6cd, State: Initialized, Role: FOLLOWER
I20260811 19:12:14.599107 17978 consensus_queue.cc:260] T 7716f27ea089411aa4aa4dd223ae7d69 P 8b314dd688dd4682b73c26337f53a6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 } } peers { permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 } } peers { permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 } }
I20260811 19:12:14.603236 17978 ts_tablet_manager.cc:1428] T 7716f27ea089411aa4aa4dd223ae7d69 P 8b314dd688dd4682b73c26337f53a6cd: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260811 19:12:14.607637 17975 tablet_bootstrap.cc:492] T 7716f27ea089411aa4aa4dd223ae7d69 P 7f82b1726e1d4ee7849c8692b83c0035: No bootstrap required, opened a new log
I20260811 19:12:14.607923 17975 ts_tablet_manager.cc:1397] T 7716f27ea089411aa4aa4dd223ae7d69 P 7f82b1726e1d4ee7849c8692b83c0035: Time spent bootstrapping tablet: real 0.022s user 0.005s sys 0.010s
I20260811 19:12:14.608125 17977 tablet_bootstrap.cc:492] T 7716f27ea089411aa4aa4dd223ae7d69 P 2ea91336741c42ae9f80f61152e39ad9: No bootstrap required, opened a new log
I20260811 19:12:14.608307 17977 ts_tablet_manager.cc:1397] T 7716f27ea089411aa4aa4dd223ae7d69 P 2ea91336741c42ae9f80f61152e39ad9: Time spent bootstrapping tablet: real 0.019s user 0.010s sys 0.001s
I20260811 19:12:14.608830 17975 raft_consensus.cc:348] T 7716f27ea089411aa4aa4dd223ae7d69 P 7f82b1726e1d4ee7849c8692b83c0035 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 } } peers { permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 } } peers { permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 } }
I20260811 19:12:14.609061 17975 raft_consensus.cc:374] T 7716f27ea089411aa4aa4dd223ae7d69 P 7f82b1726e1d4ee7849c8692b83c0035 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:12:14.609215 17977 raft_consensus.cc:348] T 7716f27ea089411aa4aa4dd223ae7d69 P 2ea91336741c42ae9f80f61152e39ad9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 } } peers { permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 } } peers { permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 } }
I20260811 19:12:14.609539 17977 raft_consensus.cc:374] T 7716f27ea089411aa4aa4dd223ae7d69 P 2ea91336741c42ae9f80f61152e39ad9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:12:14.609417 17975 raft_consensus.cc:729] T 7716f27ea089411aa4aa4dd223ae7d69 P 7f82b1726e1d4ee7849c8692b83c0035 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f82b1726e1d4ee7849c8692b83c0035, State: Initialized, Role: FOLLOWER
I20260811 19:12:14.609685 17977 raft_consensus.cc:729] T 7716f27ea089411aa4aa4dd223ae7d69 P 2ea91336741c42ae9f80f61152e39ad9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ea91336741c42ae9f80f61152e39ad9, State: Initialized, Role: FOLLOWER
I20260811 19:12:14.609920 17975 consensus_queue.cc:260] T 7716f27ea089411aa4aa4dd223ae7d69 P 7f82b1726e1d4ee7849c8692b83c0035 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 } } peers { permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 } } peers { permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 } }
I20260811 19:12:14.610046 17977 consensus_queue.cc:260] T 7716f27ea089411aa4aa4dd223ae7d69 P 2ea91336741c42ae9f80f61152e39ad9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 } } peers { permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 } } peers { permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 } }
I20260811 19:12:14.610682 17977 ts_tablet_manager.cc:1428] T 7716f27ea089411aa4aa4dd223ae7d69 P 2ea91336741c42ae9f80f61152e39ad9: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 19:12:14.612627 17975 ts_tablet_manager.cc:1428] T 7716f27ea089411aa4aa4dd223ae7d69 P 7f82b1726e1d4ee7849c8692b83c0035: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260811 19:12:14.648531 18005 raft_consensus.cc:482] T 7716f27ea089411aa4aa4dd223ae7d69 P 7f82b1726e1d4ee7849c8692b83c0035 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:12:14.648854 18005 raft_consensus.cc:504] T 7716f27ea089411aa4aa4dd223ae7d69 P 7f82b1726e1d4ee7849c8692b83c0035 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 } } peers { permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 } } peers { permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 } }
I20260811 19:12:14.650038 18005 leader_election.cc:288] T 7716f27ea089411aa4aa4dd223ae7d69 P 7f82b1726e1d4ee7849c8692b83c0035 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b314dd688dd4682b73c26337f53a6cd (127.3.61.129:45885), 2ea91336741c42ae9f80f61152e39ad9 (127.3.61.131:36703)
I20260811 19:12:14.650563 17402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7716f27ea089411aa4aa4dd223ae7d69" candidate_uuid: "7f82b1726e1d4ee7849c8692b83c0035" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b314dd688dd4682b73c26337f53a6cd" is_pre_election: true
I20260811 19:12:14.650656 17856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7716f27ea089411aa4aa4dd223ae7d69" candidate_uuid: "7f82b1726e1d4ee7849c8692b83c0035" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ea91336741c42ae9f80f61152e39ad9" is_pre_election: true
I20260811 19:12:14.650862 17402 raft_consensus.cc:2454] T 7716f27ea089411aa4aa4dd223ae7d69 P 8b314dd688dd4682b73c26337f53a6cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f82b1726e1d4ee7849c8692b83c0035 in term 0.
I20260811 19:12:14.651005 17856 raft_consensus.cc:2454] T 7716f27ea089411aa4aa4dd223ae7d69 P 2ea91336741c42ae9f80f61152e39ad9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f82b1726e1d4ee7849c8692b83c0035 in term 0.
I20260811 19:12:14.651391 17511 leader_election.cc:302] T 7716f27ea089411aa4aa4dd223ae7d69 P 7f82b1726e1d4ee7849c8692b83c0035 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f82b1726e1d4ee7849c8692b83c0035, 8b314dd688dd4682b73c26337f53a6cd; no voters:
I20260811 19:12:14.651825 18005 raft_consensus.cc:2781] T 7716f27ea089411aa4aa4dd223ae7d69 P 7f82b1726e1d4ee7849c8692b83c0035 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:12:14.651975 18005 raft_consensus.cc:482] T 7716f27ea089411aa4aa4dd223ae7d69 P 7f82b1726e1d4ee7849c8692b83c0035 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:12:14.652082 18005 raft_consensus.cc:3037] T 7716f27ea089411aa4aa4dd223ae7d69 P 7f82b1726e1d4ee7849c8692b83c0035 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:12:14.655212 18005 raft_consensus.cc:504] T 7716f27ea089411aa4aa4dd223ae7d69 P 7f82b1726e1d4ee7849c8692b83c0035 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 } } peers { permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 } } peers { permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 } }
I20260811 19:12:14.656023 18005 leader_election.cc:288] T 7716f27ea089411aa4aa4dd223ae7d69 P 7f82b1726e1d4ee7849c8692b83c0035 [CANDIDATE]: Term 1 election: Requested vote from peers 8b314dd688dd4682b73c26337f53a6cd (127.3.61.129:45885), 2ea91336741c42ae9f80f61152e39ad9 (127.3.61.131:36703)
I20260811 19:12:14.656785 17856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7716f27ea089411aa4aa4dd223ae7d69" candidate_uuid: "7f82b1726e1d4ee7849c8692b83c0035" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ea91336741c42ae9f80f61152e39ad9"
I20260811 19:12:14.656816 17402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7716f27ea089411aa4aa4dd223ae7d69" candidate_uuid: "7f82b1726e1d4ee7849c8692b83c0035" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b314dd688dd4682b73c26337f53a6cd"
I20260811 19:12:14.657016 17856 raft_consensus.cc:3037] T 7716f27ea089411aa4aa4dd223ae7d69 P 2ea91336741c42ae9f80f61152e39ad9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:12:14.657053 17402 raft_consensus.cc:3037] T 7716f27ea089411aa4aa4dd223ae7d69 P 8b314dd688dd4682b73c26337f53a6cd [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:12:14.659487 17856 raft_consensus.cc:2454] T 7716f27ea089411aa4aa4dd223ae7d69 P 2ea91336741c42ae9f80f61152e39ad9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f82b1726e1d4ee7849c8692b83c0035 in term 1.
I20260811 19:12:14.659991 17512 leader_election.cc:302] T 7716f27ea089411aa4aa4dd223ae7d69 P 7f82b1726e1d4ee7849c8692b83c0035 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ea91336741c42ae9f80f61152e39ad9, 7f82b1726e1d4ee7849c8692b83c0035; no voters:
I20260811 19:12:14.660116 17402 raft_consensus.cc:2454] T 7716f27ea089411aa4aa4dd223ae7d69 P 8b314dd688dd4682b73c26337f53a6cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f82b1726e1d4ee7849c8692b83c0035 in term 1.
I20260811 19:12:14.660320 18005 raft_consensus.cc:2781] T 7716f27ea089411aa4aa4dd223ae7d69 P 7f82b1726e1d4ee7849c8692b83c0035 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:12:14.660768 18005 raft_consensus.cc:686] T 7716f27ea089411aa4aa4dd223ae7d69 P 7f82b1726e1d4ee7849c8692b83c0035 [term 1 LEADER]: Becoming Leader. State: Replica: 7f82b1726e1d4ee7849c8692b83c0035, State: Running, Role: LEADER
I20260811 19:12:14.661096 18005 consensus_queue.cc:237] T 7716f27ea089411aa4aa4dd223ae7d69 P 7f82b1726e1d4ee7849c8692b83c0035 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 } } peers { permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 } } peers { permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 } }
I20260811 19:12:14.664144 17131 catalog_manager.cc:5495] T 7716f27ea089411aa4aa4dd223ae7d69 P 7f82b1726e1d4ee7849c8692b83c0035 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7f82b1726e1d4ee7849c8692b83c0035 (127.3.61.130). New cstate: current_term: 1 leader_uuid: "7f82b1726e1d4ee7849c8692b83c0035" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 } health_report { overall_health: UNKNOWN } } }
I20260811 19:12:14.692638 18089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:14.686674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59554 (local address 127.3.61.130:45219)
0811 19:12:14.687188 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:14.687192 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:14.687209 (+ 17us) server_negotiation.cc:407] Connection header received
0811 19:12:14.687270 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:14.687273 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:14.687336 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:14.687416 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:12:14.687871 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:14.688638 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:14.690660 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:14.691095 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:14.691195 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:14.691747 (+ 552us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:12:14.691894 (+ 147us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:12:14.691973 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:14.692177 (+ 204us) server_negotiation.cc:299] Negotiation successful
0811 19:12:14.692255 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":275,"threads_started":1}
I20260811 19:12:14.695660 17856 raft_consensus.cc:1261] T 7716f27ea089411aa4aa4dd223ae7d69 P 2ea91336741c42ae9f80f61152e39ad9 [term 1 FOLLOWER]: Refusing update from remote peer 7f82b1726e1d4ee7849c8692b83c0035: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:12:14.695659 17402 raft_consensus.cc:1261] T 7716f27ea089411aa4aa4dd223ae7d69 P 8b314dd688dd4682b73c26337f53a6cd [term 1 FOLLOWER]: Refusing update from remote peer 7f82b1726e1d4ee7849c8692b83c0035: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:12:14.696290 18085 consensus_queue.cc:1035] T 7716f27ea089411aa4aa4dd223ae7d69 P 7f82b1726e1d4ee7849c8692b83c0035 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:12:14.696681 18005 consensus_queue.cc:1035] T 7716f27ea089411aa4aa4dd223ae7d69 P 7f82b1726e1d4ee7849c8692b83c0035 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:12:14.722225 17135 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42728:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:12:14.722620 17135 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42728:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:12:14.726002 17135 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 21afe580e560414d97610b190fba81cd: Sending DeleteTablet for 3 replicas of tablet 7716f27ea089411aa4aa4dd223ae7d69
I20260811 19:12:14.726979 17375 tablet_service.cc:1514] Processing DeleteTablet for tablet 7716f27ea089411aa4aa4dd223ae7d69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:12:14 UTC) from {username='oryx', principal='oryx/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:41170
I20260811 19:12:14.726979 17597 tablet_service.cc:1514] Processing DeleteTablet for tablet 7716f27ea089411aa4aa4dd223ae7d69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:12:14 UTC) from {username='oryx', principal='oryx/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:59546
I20260811 19:12:14.727295 18012 tablet_replica.cc:330] T 7716f27ea089411aa4aa4dd223ae7d69 P 8b314dd688dd4682b73c26337f53a6cd: stopping tablet replica
I20260811 19:12:14.727294 18011 tablet_replica.cc:330] T 7716f27ea089411aa4aa4dd223ae7d69 P 7f82b1726e1d4ee7849c8692b83c0035: stopping tablet replica
I20260811 19:12:14.727312 17824 tablet_service.cc:1514] Processing DeleteTablet for tablet 7716f27ea089411aa4aa4dd223ae7d69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:12:14 UTC) from {username='oryx', principal='oryx/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:54366
I20260811 19:12:14.727479 18011 raft_consensus.cc:2229] T 7716f27ea089411aa4aa4dd223ae7d69 P 7f82b1726e1d4ee7849c8692b83c0035 [term 1 LEADER]: Raft consensus shutting down.
I20260811 19:12:14.727479 18012 raft_consensus.cc:2229] T 7716f27ea089411aa4aa4dd223ae7d69 P 8b314dd688dd4682b73c26337f53a6cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:12:14.727603 18013 tablet_replica.cc:330] T 7716f27ea089411aa4aa4dd223ae7d69 P 2ea91336741c42ae9f80f61152e39ad9: stopping tablet replica
I20260811 19:12:14.727716 18012 raft_consensus.cc:2258] T 7716f27ea089411aa4aa4dd223ae7d69 P 8b314dd688dd4682b73c26337f53a6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:12:14.727783 18013 raft_consensus.cc:2229] T 7716f27ea089411aa4aa4dd223ae7d69 P 2ea91336741c42ae9f80f61152e39ad9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:12:14.727809 18011 raft_consensus.cc:2258] T 7716f27ea089411aa4aa4dd223ae7d69 P 7f82b1726e1d4ee7849c8692b83c0035 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:12:14.728013 18013 raft_consensus.cc:2258] T 7716f27ea089411aa4aa4dd223ae7d69 P 2ea91336741c42ae9f80f61152e39ad9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:12:14.729027 18012 ts_tablet_manager.cc:1905] T 7716f27ea089411aa4aa4dd223ae7d69 P 8b314dd688dd4682b73c26337f53a6cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:12:14.729063 18013 ts_tablet_manager.cc:1905] T 7716f27ea089411aa4aa4dd223ae7d69 P 2ea91336741c42ae9f80f61152e39ad9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:12:14.729321 18011 ts_tablet_manager.cc:1905] T 7716f27ea089411aa4aa4dd223ae7d69 P 7f82b1726e1d4ee7849c8692b83c0035: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:12:14.734325 17135 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42728:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 19:12:14.735057 17135 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:12:14.736078 18012 ts_tablet_manager.cc:1918] T 7716f27ea089411aa4aa4dd223ae7d69 P 8b314dd688dd4682b73c26337f53a6cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:12:14.736222 18012 log.cc:1196] T 7716f27ea089411aa4aa4dd223ae7d69 P 8b314dd688dd4682b73c26337f53a6cd: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-0/wal/wals/7716f27ea089411aa4aa4dd223ae7d69
I20260811 19:12:14.736224 18013 ts_tablet_manager.cc:1918] T 7716f27ea089411aa4aa4dd223ae7d69 P 2ea91336741c42ae9f80f61152e39ad9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:12:14.736354 18013 log.cc:1196] T 7716f27ea089411aa4aa4dd223ae7d69 P 2ea91336741c42ae9f80f61152e39ad9: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-2/wal/wals/7716f27ea089411aa4aa4dd223ae7d69
I20260811 19:12:14.736650 18012 ts_tablet_manager.cc:1939] T 7716f27ea089411aa4aa4dd223ae7d69 P 8b314dd688dd4682b73c26337f53a6cd: Deleting consensus metadata
I20260811 19:12:14.736809 18013 ts_tablet_manager.cc:1939] T 7716f27ea089411aa4aa4dd223ae7d69 P 2ea91336741c42ae9f80f61152e39ad9: Deleting consensus metadata
I20260811 19:12:14.737989 17110 catalog_manager.cc:4841] TS 2ea91336741c42ae9f80f61152e39ad9 (127.3.61.131:36703): tablet 7716f27ea089411aa4aa4dd223ae7d69 (table test-table [id=ca3e1b226c8a4f10873c484162816632]) successfully deleted
I20260811 19:12:14.738144 17109 catalog_manager.cc:4841] TS 8b314dd688dd4682b73c26337f53a6cd (127.3.61.129:45885): tablet 7716f27ea089411aa4aa4dd223ae7d69 (table test-table [id=ca3e1b226c8a4f10873c484162816632]) successfully deleted
I20260811 19:12:14.740710 18011 ts_tablet_manager.cc:1918] T 7716f27ea089411aa4aa4dd223ae7d69 P 7f82b1726e1d4ee7849c8692b83c0035: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:12:14.740883 18011 log.cc:1196] T 7716f27ea089411aa4aa4dd223ae7d69 P 7f82b1726e1d4ee7849c8692b83c0035: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-1/wal/wals/7716f27ea089411aa4aa4dd223ae7d69
I20260811 19:12:14.741304 18011 ts_tablet_manager.cc:1939] T 7716f27ea089411aa4aa4dd223ae7d69 P 7f82b1726e1d4ee7849c8692b83c0035: Deleting consensus metadata
I20260811 19:12:14.742820 17111 catalog_manager.cc:4841] TS 7f82b1726e1d4ee7849c8692b83c0035 (127.3.61.130:45219): tablet 7716f27ea089411aa4aa4dd223ae7d69 (table test-table [id=ca3e1b226c8a4f10873c484162816632]) successfully deleted
I20260811 19:12:14.745294 17375 tablet_service.cc:1467] Processing CreateTablet for tablet dc223067e39d43339338828b13163018 (DEFAULT_TABLE table=test-table [id=35d32a97bc964234b48dced3dbacd4ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:12:14.745482 17597 tablet_service.cc:1467] Processing CreateTablet for tablet dc223067e39d43339338828b13163018 (DEFAULT_TABLE table=test-table [id=35d32a97bc964234b48dced3dbacd4ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:12:14.745795 17375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc223067e39d43339338828b13163018. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:12:14.745978 17597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc223067e39d43339338828b13163018. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:12:14.751125 17824 tablet_service.cc:1467] Processing CreateTablet for tablet dc223067e39d43339338828b13163018 (DEFAULT_TABLE table=test-table [id=35d32a97bc964234b48dced3dbacd4ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:12:14.751677 17824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc223067e39d43339338828b13163018. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:12:14.752420 17978 tablet_bootstrap.cc:492] T dc223067e39d43339338828b13163018 P 8b314dd688dd4682b73c26337f53a6cd: Bootstrap starting.
I20260811 19:12:14.754266 17978 tablet_bootstrap.cc:654] T dc223067e39d43339338828b13163018 P 8b314dd688dd4682b73c26337f53a6cd: Neither blocks nor log segments found. Creating new log.
I20260811 19:12:14.756287 17975 tablet_bootstrap.cc:492] T dc223067e39d43339338828b13163018 P 7f82b1726e1d4ee7849c8692b83c0035: Bootstrap starting.
I20260811 19:12:14.758513 17977 tablet_bootstrap.cc:492] T dc223067e39d43339338828b13163018 P 2ea91336741c42ae9f80f61152e39ad9: Bootstrap starting.
I20260811 19:12:14.758888 17975 tablet_bootstrap.cc:654] T dc223067e39d43339338828b13163018 P 7f82b1726e1d4ee7849c8692b83c0035: Neither blocks nor log segments found. Creating new log.
I20260811 19:12:14.760438 17977 tablet_bootstrap.cc:654] T dc223067e39d43339338828b13163018 P 2ea91336741c42ae9f80f61152e39ad9: Neither blocks nor log segments found. Creating new log.
I20260811 19:12:14.765153 17975 tablet_bootstrap.cc:492] T dc223067e39d43339338828b13163018 P 7f82b1726e1d4ee7849c8692b83c0035: No bootstrap required, opened a new log
I20260811 19:12:14.765370 17975 ts_tablet_manager.cc:1397] T dc223067e39d43339338828b13163018 P 7f82b1726e1d4ee7849c8692b83c0035: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.004s
I20260811 19:12:14.765924 17977 tablet_bootstrap.cc:492] T dc223067e39d43339338828b13163018 P 2ea91336741c42ae9f80f61152e39ad9: No bootstrap required, opened a new log
I20260811 19:12:14.766119 17977 ts_tablet_manager.cc:1397] T dc223067e39d43339338828b13163018 P 2ea91336741c42ae9f80f61152e39ad9: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.005s
I20260811 19:12:14.766371 17975 raft_consensus.cc:348] T dc223067e39d43339338828b13163018 P 7f82b1726e1d4ee7849c8692b83c0035 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 } } peers { permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 } } peers { permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 } }
I20260811 19:12:14.766630 17975 raft_consensus.cc:374] T dc223067e39d43339338828b13163018 P 7f82b1726e1d4ee7849c8692b83c0035 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:12:14.766712 17975 raft_consensus.cc:729] T dc223067e39d43339338828b13163018 P 7f82b1726e1d4ee7849c8692b83c0035 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f82b1726e1d4ee7849c8692b83c0035, State: Initialized, Role: FOLLOWER
I20260811 19:12:14.766918 17977 raft_consensus.cc:348] T dc223067e39d43339338828b13163018 P 2ea91336741c42ae9f80f61152e39ad9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 } } peers { permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 } } peers { permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 } }
I20260811 19:12:14.766987 17975 consensus_queue.cc:260] T dc223067e39d43339338828b13163018 P 7f82b1726e1d4ee7849c8692b83c0035 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 } } peers { permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 } } peers { permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 } }
I20260811 19:12:14.767201 17977 raft_consensus.cc:374] T dc223067e39d43339338828b13163018 P 2ea91336741c42ae9f80f61152e39ad9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:12:14.767287 17977 raft_consensus.cc:729] T dc223067e39d43339338828b13163018 P 2ea91336741c42ae9f80f61152e39ad9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ea91336741c42ae9f80f61152e39ad9, State: Initialized, Role: FOLLOWER
I20260811 19:12:14.767692 17977 consensus_queue.cc:260] T dc223067e39d43339338828b13163018 P 2ea91336741c42ae9f80f61152e39ad9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 } } peers { permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 } } peers { permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 } }
I20260811 19:12:14.768503 17978 tablet_bootstrap.cc:492] T dc223067e39d43339338828b13163018 P 8b314dd688dd4682b73c26337f53a6cd: No bootstrap required, opened a new log
I20260811 19:12:14.768778 17978 ts_tablet_manager.cc:1397] T dc223067e39d43339338828b13163018 P 8b314dd688dd4682b73c26337f53a6cd: Time spent bootstrapping tablet: real 0.016s user 0.008s sys 0.004s
I20260811 19:12:14.769738 17975 ts_tablet_manager.cc:1428] T dc223067e39d43339338828b13163018 P 7f82b1726e1d4ee7849c8692b83c0035: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260811 19:12:14.769770 17978 raft_consensus.cc:348] T dc223067e39d43339338828b13163018 P 8b314dd688dd4682b73c26337f53a6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 } } peers { permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 } } peers { permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 } }
I20260811 19:12:14.770427 17978 raft_consensus.cc:374] T dc223067e39d43339338828b13163018 P 8b314dd688dd4682b73c26337f53a6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:12:14.770570 17978 raft_consensus.cc:729] T dc223067e39d43339338828b13163018 P 8b314dd688dd4682b73c26337f53a6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b314dd688dd4682b73c26337f53a6cd, State: Initialized, Role: FOLLOWER
I20260811 19:12:14.770891 17978 consensus_queue.cc:260] T dc223067e39d43339338828b13163018 P 8b314dd688dd4682b73c26337f53a6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 } } peers { permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 } } peers { permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 } }
I20260811 19:12:14.771509 17978 ts_tablet_manager.cc:1428] T dc223067e39d43339338828b13163018 P 8b314dd688dd4682b73c26337f53a6cd: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 19:12:14.772776 17977 ts_tablet_manager.cc:1428] T dc223067e39d43339338828b13163018 P 2ea91336741c42ae9f80f61152e39ad9: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260811 19:12:14.894713 17984 raft_consensus.cc:482] T dc223067e39d43339338828b13163018 P 8b314dd688dd4682b73c26337f53a6cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:12:14.894979 17984 raft_consensus.cc:504] T dc223067e39d43339338828b13163018 P 8b314dd688dd4682b73c26337f53a6cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 } } peers { permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 } } peers { permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 } }
I20260811 19:12:14.896787 17984 leader_election.cc:288] T dc223067e39d43339338828b13163018 P 8b314dd688dd4682b73c26337f53a6cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f82b1726e1d4ee7849c8692b83c0035 (127.3.61.130:45219), 2ea91336741c42ae9f80f61152e39ad9 (127.3.61.131:36703)
I20260811 19:12:14.917151 18089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:14.897418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:58973 (local address 127.3.61.130:45219)
0811 19:12:14.898083 (+ 665us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:14.898091 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:14.908227 (+ 10136us) server_negotiation.cc:407] Connection header received
0811 19:12:14.908805 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:14.908812 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:14.908916 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:14.908986 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:12:14.909968 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:14.911243 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:14.914452 (+ 3209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:14.915853 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:14.915987 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:14.916068 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:14.916665 (+ 597us) server_negotiation.cc:299] Negotiation successful
0811 19:12:14.916842 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159}
I20260811 19:12:14.920042 18100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:14.897005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.130:45219 (local address 127.3.61.129:58973)
0811 19:12:14.908060 (+ 11055us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:14.908089 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:14.908613 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:14.909221 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:14.909226 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:14.909257 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:12:14.909712 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:14.909727 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:14.911584 (+ 1857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:14.911591 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:14.914213 (+ 2622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:14.914231 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:14.916216 (+ 1985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:14.916222 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:14.916431 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:14.916482 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 19:12:14.919372 (+ 2890us) client_negotiation.cc:240] Negotiation successful
0811 19:12:14.919546 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10862,"mutex_wait_us":1,"thread_start_us":11242,"threads_started":1}
I20260811 19:12:14.920921 18103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:14.897821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:37005 (local address 127.3.61.131:36703)
0811 19:12:14.902113 (+ 4292us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:14.902120 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:14.912823 (+ 10703us) server_negotiation.cc:407] Connection header received
0811 19:12:14.913285 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:14.913292 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:14.913400 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:14.913475 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:12:14.914458 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:14.915734 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:14.918206 (+ 2472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:14.919570 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:14.919706 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:14.919783 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:14.920431 (+ 648us) server_negotiation.cc:299] Negotiation successful
0811 19:12:14.920585 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4138,"thread_start_us":3980,"threads_started":1}
I20260811 19:12:14.920989 17625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc223067e39d43339338828b13163018" candidate_uuid: "8b314dd688dd4682b73c26337f53a6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f82b1726e1d4ee7849c8692b83c0035" is_pre_election: true
I20260811 19:12:14.921259 17625 raft_consensus.cc:2454] T dc223067e39d43339338828b13163018 P 7f82b1726e1d4ee7849c8692b83c0035 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b314dd688dd4682b73c26337f53a6cd in term 0.
I20260811 19:12:14.921739 18101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:14.897232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.131:36703 (local address 127.3.61.129:37005)
0811 19:12:14.912706 (+ 15474us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:14.912727 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:14.912942 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:14.913796 (+ 854us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:14.913800 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:14.913829 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:12:14.914247 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:14.914258 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:14.915955 (+ 1697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:14.915960 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:14.917966 (+ 2006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:14.917984 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:14.920011 (+ 2027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:14.920019 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:14.920225 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:14.920256 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 19:12:14.921190 (+ 934us) client_negotiation.cc:240] Negotiation successful
0811 19:12:14.921359 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15024,"mutex_wait_us":2,"thread_start_us":15091,"threads_started":1}
I20260811 19:12:14.922235 17303 leader_election.cc:302] T dc223067e39d43339338828b13163018 P 8b314dd688dd4682b73c26337f53a6cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f82b1726e1d4ee7849c8692b83c0035, 8b314dd688dd4682b73c26337f53a6cd; no voters:
I20260811 19:12:14.922428 17856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc223067e39d43339338828b13163018" candidate_uuid: "8b314dd688dd4682b73c26337f53a6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ea91336741c42ae9f80f61152e39ad9" is_pre_election: true
I20260811 19:12:14.922677 17856 raft_consensus.cc:2454] T dc223067e39d43339338828b13163018 P 2ea91336741c42ae9f80f61152e39ad9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b314dd688dd4682b73c26337f53a6cd in term 0.
I20260811 19:12:14.922737 17984 raft_consensus.cc:2781] T dc223067e39d43339338828b13163018 P 8b314dd688dd4682b73c26337f53a6cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:12:14.922857 17984 raft_consensus.cc:482] T dc223067e39d43339338828b13163018 P 8b314dd688dd4682b73c26337f53a6cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:12:14.922947 17984 raft_consensus.cc:3037] T dc223067e39d43339338828b13163018 P 8b314dd688dd4682b73c26337f53a6cd [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:12:14.925601 17984 raft_consensus.cc:504] T dc223067e39d43339338828b13163018 P 8b314dd688dd4682b73c26337f53a6cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 } } peers { permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 } } peers { permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 } }
I20260811 19:12:14.926992 17984 leader_election.cc:288] T dc223067e39d43339338828b13163018 P 8b314dd688dd4682b73c26337f53a6cd [CANDIDATE]: Term 1 election: Requested vote from peers 7f82b1726e1d4ee7849c8692b83c0035 (127.3.61.130:45219), 2ea91336741c42ae9f80f61152e39ad9 (127.3.61.131:36703)
I20260811 19:12:14.927577 17625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc223067e39d43339338828b13163018" candidate_uuid: "8b314dd688dd4682b73c26337f53a6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f82b1726e1d4ee7849c8692b83c0035"
I20260811 19:12:14.927599 17856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc223067e39d43339338828b13163018" candidate_uuid: "8b314dd688dd4682b73c26337f53a6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ea91336741c42ae9f80f61152e39ad9"
I20260811 19:12:14.927824 17625 raft_consensus.cc:3037] T dc223067e39d43339338828b13163018 P 7f82b1726e1d4ee7849c8692b83c0035 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:12:14.927832 17856 raft_consensus.cc:3037] T dc223067e39d43339338828b13163018 P 2ea91336741c42ae9f80f61152e39ad9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:12:14.931061 17856 raft_consensus.cc:2454] T dc223067e39d43339338828b13163018 P 2ea91336741c42ae9f80f61152e39ad9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b314dd688dd4682b73c26337f53a6cd in term 1.
I20260811 19:12:14.931327 17625 raft_consensus.cc:2454] T dc223067e39d43339338828b13163018 P 7f82b1726e1d4ee7849c8692b83c0035 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b314dd688dd4682b73c26337f53a6cd in term 1.
I20260811 19:12:14.931645 17302 leader_election.cc:302] T dc223067e39d43339338828b13163018 P 8b314dd688dd4682b73c26337f53a6cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ea91336741c42ae9f80f61152e39ad9, 8b314dd688dd4682b73c26337f53a6cd; no voters:
I20260811 19:12:14.932049 17984 raft_consensus.cc:2781] T dc223067e39d43339338828b13163018 P 8b314dd688dd4682b73c26337f53a6cd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:12:14.932521 17984 raft_consensus.cc:686] T dc223067e39d43339338828b13163018 P 8b314dd688dd4682b73c26337f53a6cd [term 1 LEADER]: Becoming Leader. State: Replica: 8b314dd688dd4682b73c26337f53a6cd, State: Running, Role: LEADER
I20260811 19:12:14.932914 17984 consensus_queue.cc:237] T dc223067e39d43339338828b13163018 P 8b314dd688dd4682b73c26337f53a6cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 } } peers { permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 } } peers { permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 } }
I20260811 19:12:14.937230 17131 catalog_manager.cc:5495] T dc223067e39d43339338828b13163018 P 8b314dd688dd4682b73c26337f53a6cd reported cstate change: term changed from 0 to 1, leader changed from <none> to 8b314dd688dd4682b73c26337f53a6cd (127.3.61.129). New cstate: current_term: 1 leader_uuid: "8b314dd688dd4682b73c26337f53a6cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b314dd688dd4682b73c26337f53a6cd" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 45885 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 } health_report { overall_health: UNKNOWN } } }
I20260811 19:12:14.980333 18112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:14.973308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41184 (local address 127.3.61.129:45885)
0811 19:12:14.973942 (+ 634us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:14.973946 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:14.973968 (+ 22us) server_negotiation.cc:407] Connection header received
0811 19:12:14.974064 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:14.974067 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:14.974147 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:14.974301 (+ 154us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:12:14.974676 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:14.975455 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:14.977361 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:14.978056 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:14.978163 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:14.978878 (+ 715us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:12:14.979558 (+ 680us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:12:14.979672 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:14.979869 (+ 197us) server_negotiation.cc:299] Negotiation successful
0811 19:12:14.979956 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":513,"thread_start_us":351,"threads_started":1}
I20260811 19:12:15.000545 18103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:14.993570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:44557 (local address 127.3.61.131:36703)
0811 19:12:14.993829 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:14.993844 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:14.994483 (+ 639us) server_negotiation.cc:407] Connection header received
0811 19:12:14.994631 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:14.994636 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:14.994731 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:14.994862 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:12:14.995539 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:14.996693 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:14.998248 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:14.998921 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:14.999060 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:14.999597 (+ 537us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:12:14.999775 (+ 178us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:12:14.999896 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:15.000115 (+ 219us) server_negotiation.cc:299] Negotiation successful
0811 19:12:15.000207 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260811 19:12:15.000631 18117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:14.993518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.131:36703 (local address 127.3.61.129:44557)
0811 19:12:14.994341 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:14.994359 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:14.994500 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:14.994952 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:14.994957 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:14.994972 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 19:12:14.995333 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:14.995345 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:14.996882 (+ 1537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:14.996887 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:14.998096 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:14.998110 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:14.999137 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:14.999142 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:14.999280 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:14.999360 (+ 80us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 19:12:14.999941 (+ 581us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 19:12:14.999963 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 19:12:15.000067 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 19:12:15.000219 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":668,"thread_start_us":422,"threads_started":1}
I20260811 19:12:15.012454 17856 raft_consensus.cc:1261] T dc223067e39d43339338828b13163018 P 2ea91336741c42ae9f80f61152e39ad9 [term 1 FOLLOWER]: Refusing update from remote peer 8b314dd688dd4682b73c26337f53a6cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:12:15.012578 17625 raft_consensus.cc:1261] T dc223067e39d43339338828b13163018 P 7f82b1726e1d4ee7849c8692b83c0035 [term 1 FOLLOWER]: Refusing update from remote peer 8b314dd688dd4682b73c26337f53a6cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:12:15.013356 18108 consensus_queue.cc:1035] T dc223067e39d43339338828b13163018 P 8b314dd688dd4682b73c26337f53a6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f82b1726e1d4ee7849c8692b83c0035" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:12:15.013787 17984 consensus_queue.cc:1035] T dc223067e39d43339338828b13163018 P 8b314dd688dd4682b73c26337f53a6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ea91336741c42ae9f80f61152e39ad9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:12:15.048935 18112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:15.042769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:35033 (local address 127.3.61.129:45885)
0811 19:12:15.043120 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:15.043127 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:15.043149 (+ 22us) server_negotiation.cc:407] Connection header received
0811 19:12:15.043291 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:15.043296 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:15.043382 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:15.043488 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:12:15.044296 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:15.045326 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:15.047092 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:15.047579 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:15.047698 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:15.048143 (+ 445us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:12:15.048276 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:12:15.048363 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:15.048501 (+ 138us) server_negotiation.cc:299] Negotiation successful
0811 19:12:15.048586 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 19:12:15.049010 18125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:15.042280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:45885 (local address 127.3.61.131:35033)
0811 19:12:15.043024 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:15.043045 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:15.043151 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:15.043767 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:15.043774 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:15.043788 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 19:12:15.044133 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:15.044143 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:15.045463 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:15.045469 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:15.046934 (+ 1465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:15.046949 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:15.047794 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:15.047800 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:15.047936 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:15.047977 (+ 41us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 19:12:15.048380 (+ 403us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 19:12:15.048393 (+ 13us) client_negotiation.cc:769] Sending connection context
0811 19:12:15.048467 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 19:12:15.048616 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":385,"threads_started":1}
I20260811 19:12:15.095717 17131 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42728:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:12:15.096060 17131 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42728:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:12:15.100888 17131 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 21afe580e560414d97610b190fba81cd: Sending DeleteTablet for 3 replicas of tablet dc223067e39d43339338828b13163018
I20260811 19:12:15.102406 17824 tablet_service.cc:1514] Processing DeleteTablet for tablet dc223067e39d43339338828b13163018 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:12:15 UTC) from {username='oryx', principal='oryx/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:54366
I20260811 19:12:15.102545 17375 tablet_service.cc:1514] Processing DeleteTablet for tablet dc223067e39d43339338828b13163018 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:12:15 UTC) from {username='oryx', principal='oryx/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:41170
I20260811 19:12:15.102762 18013 tablet_replica.cc:330] T dc223067e39d43339338828b13163018 P 2ea91336741c42ae9f80f61152e39ad9: stopping tablet replica
I20260811 19:12:15.102999 18012 tablet_replica.cc:330] T dc223067e39d43339338828b13163018 P 8b314dd688dd4682b73c26337f53a6cd: stopping tablet replica
I20260811 19:12:15.103019 18013 raft_consensus.cc:2229] T dc223067e39d43339338828b13163018 P 2ea91336741c42ae9f80f61152e39ad9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:12:15.103259 18013 raft_consensus.cc:2258] T dc223067e39d43339338828b13163018 P 2ea91336741c42ae9f80f61152e39ad9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:12:15.103262 18012 raft_consensus.cc:2229] T dc223067e39d43339338828b13163018 P 8b314dd688dd4682b73c26337f53a6cd [term 1 LEADER]: Raft consensus shutting down.
I20260811 19:12:15.104063 18012 raft_consensus.cc:2258] T dc223067e39d43339338828b13163018 P 8b314dd688dd4682b73c26337f53a6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:12:15.104151 17597 tablet_service.cc:1514] Processing DeleteTablet for tablet dc223067e39d43339338828b13163018 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:12:15 UTC) from {username='oryx', principal='oryx/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:59546
I20260811 19:12:15.104516 18011 tablet_replica.cc:330] T dc223067e39d43339338828b13163018 P 7f82b1726e1d4ee7849c8692b83c0035: stopping tablet replica
I20260811 19:12:15.104729 18011 raft_consensus.cc:2229] T dc223067e39d43339338828b13163018 P 7f82b1726e1d4ee7849c8692b83c0035 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:12:15.104933 18011 raft_consensus.cc:2258] T dc223067e39d43339338828b13163018 P 7f82b1726e1d4ee7849c8692b83c0035 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:12:15.106369 18011 ts_tablet_manager.cc:1905] T dc223067e39d43339338828b13163018 P 7f82b1726e1d4ee7849c8692b83c0035: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:12:15.106534 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 17270
I20260811 19:12:15.110013 18013 ts_tablet_manager.cc:1905] T dc223067e39d43339338828b13163018 P 2ea91336741c42ae9f80f61152e39ad9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:12:15.111749 18012 ts_tablet_manager.cc:1905] T dc223067e39d43339338828b13163018 P 8b314dd688dd4682b73c26337f53a6cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:12:15.120428 18013 ts_tablet_manager.cc:1918] T dc223067e39d43339338828b13163018 P 2ea91336741c42ae9f80f61152e39ad9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 19:12:15.120762 18013 log.cc:1196] T dc223067e39d43339338828b13163018 P 2ea91336741c42ae9f80f61152e39ad9: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-2/wal/wals/dc223067e39d43339338828b13163018
I20260811 19:12:15.121333 18013 ts_tablet_manager.cc:1939] T dc223067e39d43339338828b13163018 P 2ea91336741c42ae9f80f61152e39ad9: Deleting consensus metadata
I20260811 19:12:15.122867 17110 catalog_manager.cc:4841] TS 2ea91336741c42ae9f80f61152e39ad9 (127.3.61.131:36703): tablet dc223067e39d43339338828b13163018 (table test-table [id=35d32a97bc964234b48dced3dbacd4ee]) successfully deleted
I20260811 19:12:15.127363 18011 ts_tablet_manager.cc:1918] T dc223067e39d43339338828b13163018 P 7f82b1726e1d4ee7849c8692b83c0035: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 19:12:15.127666 18011 log.cc:1196] T dc223067e39d43339338828b13163018 P 7f82b1726e1d4ee7849c8692b83c0035: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-1/wal/wals/dc223067e39d43339338828b13163018
I20260811 19:12:15.128180 18012 ts_tablet_manager.cc:1918] T dc223067e39d43339338828b13163018 P 8b314dd688dd4682b73c26337f53a6cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 19:12:15.128751 18011 ts_tablet_manager.cc:1939] T dc223067e39d43339338828b13163018 P 7f82b1726e1d4ee7849c8692b83c0035: Deleting consensus metadata
I20260811 19:12:15.128961 18012 log.cc:1196] T dc223067e39d43339338828b13163018 P 8b314dd688dd4682b73c26337f53a6cd: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786475454804547-3318-0/minicluster-data/ts-0/wal/wals/dc223067e39d43339338828b13163018
I20260811 19:12:15.130493 17111 catalog_manager.cc:4841] TS 7f82b1726e1d4ee7849c8692b83c0035 (127.3.61.130:45219): tablet dc223067e39d43339338828b13163018 (table test-table [id=35d32a97bc964234b48dced3dbacd4ee]) successfully deleted
I20260811 19:12:15.131114 18012 ts_tablet_manager.cc:1939] T dc223067e39d43339338828b13163018 P 8b314dd688dd4682b73c26337f53a6cd: Deleting consensus metadata
I20260811 19:12:15.133749 17109 catalog_manager.cc:4841] TS 8b314dd688dd4682b73c26337f53a6cd (127.3.61.129:45885): tablet dc223067e39d43339338828b13163018 (table test-table [id=35d32a97bc964234b48dced3dbacd4ee]) successfully deleted
Aug 11 19:12:15 dist-test-slave-t7g5 krb5kdc[17058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475534, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.61.129@KRBTEST.COM
I20260811 19:12:15.155524 18112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:15.109951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41190 (local address 127.3.61.129:45885)
0811 19:12:15.110164 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:15.110171 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:15.133461 (+ 23290us) server_negotiation.cc:407] Connection header received
0811 19:12:15.133691 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:15.133697 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:15.133796 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:15.133901 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:15.135961 (+ 2060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:15.137153 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:15.139574 (+ 2421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:15.140302 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:15.140456 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:15.146317 (+ 5861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:15.146337 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:15.146358 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:15.146419 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:15.151539 (+ 5120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:15.152656 (+ 1117us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:15.152666 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:15.152671 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:15.152881 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:15.153363 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:15.153369 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:15.153372 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:15.153947 (+ 575us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:15.154305 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:15.154801 (+ 496us) server_negotiation.cc:299] Negotiation successful
0811 19:12:15.155071 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 19:12:15.156575 17470 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:12:15.282704 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 17270
I20260811 19:12:15.314152 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 17482
Aug 11 19:12:15 dist-test-slave-t7g5 krb5kdc[17058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475534, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.61.130@KRBTEST.COM
I20260811 19:12:15.330883 18089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:15.314992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59566 (local address 127.3.61.130:45219)
0811 19:12:15.315156 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:15.315161 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:15.315180 (+ 19us) server_negotiation.cc:407] Connection header received
0811 19:12:15.315628 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:15.315632 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:15.315717 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:15.315803 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:15.317296 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:15.318336 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:15.319866 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:15.320464 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:15.320574 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:15.324977 (+ 4403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:15.324994 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:15.325014 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:15.325055 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:15.327903 (+ 2848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:15.328625 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:15.328633 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:15.328635 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:15.328779 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:15.329295 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:15.329298 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:15.329301 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:15.329761 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:15.330034 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:15.330396 (+ 362us) server_negotiation.cc:299] Negotiation successful
0811 19:12:15.330603 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 19:12:15.331660 17701 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:12:15.460170 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 17482
I20260811 19:12:15.495344 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 17713
Aug 11 19:12:15 dist-test-slave-t7g5 krb5kdc[17058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475534, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.61.131@KRBTEST.COM
I20260811 19:12:15.513511 18103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:15.496487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54396 (local address 127.3.61.131:36703)
0811 19:12:15.496697 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:15.496701 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:15.496724 (+ 23us) server_negotiation.cc:407] Connection header received
0811 19:12:15.496781 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:15.496784 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:15.496851 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:15.496952 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:15.498388 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:15.499111 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:15.500490 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:15.500930 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:15.501042 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:15.506079 (+ 5037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:15.506099 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:15.506118 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:15.506164 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:15.509208 (+ 3044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:15.510284 (+ 1076us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:15.510298 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:15.510301 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:15.510629 (+ 328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:15.511500 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:15.511508 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:15.511512 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:15.512116 (+ 604us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:15.512482 (+ 366us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:15.512831 (+ 349us) server_negotiation.cc:299] Negotiation successful
0811 19:12:15.513109 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 19:12:15.514333 17926 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:12:15.675282 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 17713
I20260811 19:12:15.711712 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 17083
I20260811 19:12:15.713222 17210 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:12:15.838636 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 17083
2026-08-11T19:12:15Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (6509 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786475454804547-3318-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:12:15 dist-test-slave-t7g5 krb5kdc[18142](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:12:15 dist-test-slave-t7g5 krb5kdc[18142](info): set up 2 sockets
Aug 11 19:12:15 dist-test-slave-t7g5 krb5kdc[18142](info): commencing operation
krb5kdc: starting...
W20260811 19:12:17.155077 3318 mini_kdc.cc:121] Time spent starting KDC: real 1.268s user 0.002s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Aug 11 19:12:17 dist-test-slave-t7g5 krb5kdc[18142](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475537, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:12:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:12:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.61.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab.
Entry for principal kudu/127.3.61.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab.
I20260811 19:12:17.478292 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786475454804547-3318-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786475454804547-3318-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786475454804547-3318-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:39599
--webserver_interface=127.3.61.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:38229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.61.190:39599
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:12:17.864215 18167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:12:17.864579 18167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:12:17.864717 18167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:12:17.872891 18167 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:12:17.873112 18167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:12:17.873183 18167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:12:17.873293 18167 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:12:17.873371 18167 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:12:17.885262 18167 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:38229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786475454804547-3318-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786475454804547-3318-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.61.190:39599
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:39599
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.61.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786475454804547-3318-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:12:17.886970 18167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:12:17.888958 18167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:12:17.903514 18174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:17.903558 18175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:17.903487 18179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:12:17.904428 18167 server_base.cc:1034] running on GCE node
I20260811 19:12:17.905356 18167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:12:17.907138 18167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:12:17.908463 18167 hybrid_clock.cc:648] HybridClock initialized: now 1786475537908375 us; error 62 us; skew 500 ppm
I20260811 19:12:17.909071 18167 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-11T19:12:18Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (2142 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:12:18 dist-test-slave-t7g5 krb5kdc[18187](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:12:18 dist-test-slave-t7g5 krb5kdc[18187](info): set up 2 sockets
Aug 11 19:12:18 dist-test-slave-t7g5 krb5kdc[18187](info): commencing operation
krb5kdc: starting...
W20260811 19:12:19.303884 3318 mini_kdc.cc:121] Time spent starting KDC: real 1.274s user 0.000s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Aug 11 19:12:19 dist-test-slave-t7g5 krb5kdc[18187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475539, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:12:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:12:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.61.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab.
Entry for principal kudu/127.3.61.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab.
I20260811 19:12:19.623135 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:35751
--webserver_interface=127.3.61.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:34597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.61.190:35751
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:12:19.999825 18208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:12:20.000178 18208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:12:20.000293 18208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:12:20.009351 18208 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:12:20.009505 18208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:12:20.009562 18208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:12:20.009678 18208 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:12:20.009733 18208 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:12:20.020364 18208 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:34597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.61.190:35751
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:35751
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.61.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:12:20.022323 18208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:12:20.024643 18208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:12:20.041420 18216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:20.041359 18219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:20.041323 18214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:12:20.042377 18208 server_base.cc:1034] running on GCE node
I20260811 19:12:20.043607 18208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:12:20.045943 18208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:12:20.047315 18208 hybrid_clock.cc:648] HybridClock initialized: now 1786475540047233 us; error 79 us; skew 500 ppm
I20260811 19:12:20.047917 18208 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:12:20 dist-test-slave-t7g5 krb5kdc[18187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475540, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:12:20.055837 18208 init.cc:377] Logged in from keytab as kudu/127.3.61.190@KRBTEST.COM (short username kudu)
I20260811 19:12:20.058365 18208 webserver.cc:460] Webserver started at http://127.3.61.190:43267/ using document root <none> and password file <none>
I20260811 19:12:20.059406 18208 fs_manager.cc:359] Metadata directory not provided
I20260811 19:12:20.059568 18208 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:12:20.059968 18208 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:12:20.064069 18208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/master-0/data/instance:
uuid: "45cde14fb6a94f3d8f9c9031f823cc19"
format_stamp: "Formatted at 2026-08-11 19:12:20 on dist-test-slave-t7g5"
server_key: "ffd185c0a338a9d0624af287350d5936"
server_key_iv: "0427ae1d2c2c75996a6406c159b8cec0"
server_key_version: "encryptionkey@0"
I20260811 19:12:20.065191 18208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/master-0/wal/instance:
uuid: "45cde14fb6a94f3d8f9c9031f823cc19"
format_stamp: "Formatted at 2026-08-11 19:12:20 on dist-test-slave-t7g5"
server_key: "ffd185c0a338a9d0624af287350d5936"
server_key_iv: "0427ae1d2c2c75996a6406c159b8cec0"
server_key_version: "encryptionkey@0"
I20260811 19:12:20.072592 18208 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260811 19:12:20.076926 18231 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:20.078810 18208 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 19:12:20.079043 18208 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/master-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/master-0/wal
uuid: "45cde14fb6a94f3d8f9c9031f823cc19"
format_stamp: "Formatted at 2026-08-11 19:12:20 on dist-test-slave-t7g5"
server_key: "ffd185c0a338a9d0624af287350d5936"
server_key_iv: "0427ae1d2c2c75996a6406c159b8cec0"
server_key_version: "encryptionkey@0"
I20260811 19:12:20.079278 18208 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:12:20.099807 18208 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:12:20.105944 18208 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:12:20.106261 18208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:12:20.127156 18208 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.190:35751
I20260811 19:12:20.127240 18306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.190:35751 every 8 connection(s)
I20260811 19:12:20.128705 18208 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
I20260811 19:12:20.133265 18307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:12:20.136615 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 18208
I20260811 19:12:20.136866 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/master-0/wal/instance
I20260811 19:12:20.137735 3318 external_mini_cluster.cc:1428] Setting key d5fbafea891283fa4860d8ad1f27731c
I20260811 19:12:20.148289 18307 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45cde14fb6a94f3d8f9c9031f823cc19: Bootstrap starting.
I20260811 19:12:20.152642 18307 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 45cde14fb6a94f3d8f9c9031f823cc19: Neither blocks nor log segments found. Creating new log.
I20260811 19:12:20.154102 18307 log.cc:824] T 00000000000000000000000000000000 P 45cde14fb6a94f3d8f9c9031f823cc19: Log is configured to *not* fsync() on all Append() calls
Aug 11 19:12:20 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475539, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:12:20.159294 18307 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45cde14fb6a94f3d8f9c9031f823cc19: No bootstrap required, opened a new log
I20260811 19:12:20.164480 18314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:20.140283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51026 (local address 127.3.61.190:35751)
0811 19:12:20.141158 (+ 875us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:20.141170 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:20.141214 (+ 44us) server_negotiation.cc:407] Connection header received
0811 19:12:20.142995 (+ 1781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:20.143024 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:20.143625 (+ 601us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:20.144037 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:20.146423 (+ 2386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:20.147822 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:20.149626 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:20.150483 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:20.150640 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:20.155573 (+ 4933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:20.155584 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:20.155603 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:20.155658 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:20.159477 (+ 3819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:20.160454 (+ 977us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:20.160463 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:20.160471 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:20.160693 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:20.161114 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:20.161118 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:20.161121 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:20.161898 (+ 777us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:20.162252 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:20.162735 (+ 483us) server_negotiation.cc:299] Negotiation successful
0811 19:12:20.163226 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":638,"thread_start_us":369,"threads_started":1}
I20260811 19:12:20.169592 18307 raft_consensus.cc:348] T 00000000000000000000000000000000 P 45cde14fb6a94f3d8f9c9031f823cc19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45cde14fb6a94f3d8f9c9031f823cc19" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35751 } }
I20260811 19:12:20.170111 18307 raft_consensus.cc:374] T 00000000000000000000000000000000 P 45cde14fb6a94f3d8f9c9031f823cc19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:12:20.170223 18307 raft_consensus.cc:729] T 00000000000000000000000000000000 P 45cde14fb6a94f3d8f9c9031f823cc19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45cde14fb6a94f3d8f9c9031f823cc19, State: Initialized, Role: FOLLOWER
I20260811 19:12:20.170847 18307 consensus_queue.cc:260] T 00000000000000000000000000000000 P 45cde14fb6a94f3d8f9c9031f823cc19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45cde14fb6a94f3d8f9c9031f823cc19" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35751 } }
I20260811 19:12:20.171099 18307 raft_consensus.cc:388] T 00000000000000000000000000000000 P 45cde14fb6a94f3d8f9c9031f823cc19 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:12:20.171247 18307 raft_consensus.cc:482] T 00000000000000000000000000000000 P 45cde14fb6a94f3d8f9c9031f823cc19 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:12:20.171427 18307 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 45cde14fb6a94f3d8f9c9031f823cc19 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:12:20.173926 18307 raft_consensus.cc:504] T 00000000000000000000000000000000 P 45cde14fb6a94f3d8f9c9031f823cc19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45cde14fb6a94f3d8f9c9031f823cc19" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35751 } }
I20260811 19:12:20.174551 18307 leader_election.cc:302] T 00000000000000000000000000000000 P 45cde14fb6a94f3d8f9c9031f823cc19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 45cde14fb6a94f3d8f9c9031f823cc19; no voters:
I20260811 19:12:20.175187 18307 leader_election.cc:288] T 00000000000000000000000000000000 P 45cde14fb6a94f3d8f9c9031f823cc19 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:12:20.175329 18316 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 45cde14fb6a94f3d8f9c9031f823cc19 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:12:20.176039 18316 raft_consensus.cc:686] T 00000000000000000000000000000000 P 45cde14fb6a94f3d8f9c9031f823cc19 [term 1 LEADER]: Becoming Leader. State: Replica: 45cde14fb6a94f3d8f9c9031f823cc19, State: Running, Role: LEADER
I20260811 19:12:20.176638 18316 consensus_queue.cc:237] T 00000000000000000000000000000000 P 45cde14fb6a94f3d8f9c9031f823cc19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45cde14fb6a94f3d8f9c9031f823cc19" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35751 } }
I20260811 19:12:20.177382 18307 sys_catalog.cc:564] T 00000000000000000000000000000000 P 45cde14fb6a94f3d8f9c9031f823cc19 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:12:20.179628 18319 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45cde14fb6a94f3d8f9c9031f823cc19 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 45cde14fb6a94f3d8f9c9031f823cc19. Latest consensus state: current_term: 1 leader_uuid: "45cde14fb6a94f3d8f9c9031f823cc19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45cde14fb6a94f3d8f9c9031f823cc19" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35751 } } }
I20260811 19:12:20.180068 18319 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45cde14fb6a94f3d8f9c9031f823cc19 [sys.catalog]: This master's current role is: LEADER
I20260811 19:12:20.184944 18318 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45cde14fb6a94f3d8f9c9031f823cc19 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "45cde14fb6a94f3d8f9c9031f823cc19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45cde14fb6a94f3d8f9c9031f823cc19" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35751 } } }
I20260811 19:12:20.185334 18318 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45cde14fb6a94f3d8f9c9031f823cc19 [sys.catalog]: This master's current role is: LEADER
I20260811 19:12:20.186370 18327 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:12:20.192531 18327 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:12:20.206161 18327 catalog_manager.cc:1347] Generated new cluster ID: 64396ceafe71488baa6c058b98b25268
I20260811 19:12:20.206328 18327 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:12:20.217541 18327 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:12:20.219733 18327 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:12:20.238437 18327 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 45cde14fb6a94f3d8f9c9031f823cc19: Generated new TSK 0
I20260811 19:12:20.239447 18327 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.61.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab.
Entry for principal kudu/127.3.61.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab.
I20260811 19:12:20.322286 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.129:0
--local_ip_for_outbound_sockets=127.3.61.129
--webserver_interface=127.3.61.129
--webserver_port=0
--tserver_master_addrs=127.3.61.190:35751
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab
--principal=kudu/127.3.61.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:34597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:12:20.703642 18353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:12:20.703982 18353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:12:20.704128 18353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:12:20.711864 18353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:12:20.712052 18353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:12:20.712231 18353 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.129
I20260811 19:12:20.723455 18353 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:34597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab
--principal=kudu/127.3.61.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.61.129
--webserver_port=0
--tserver_master_addrs=127.3.61.190:35751
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.129
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:12:20.724923 18353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:12:20.726402 18353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:12:20.741452 18365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:20.741387 18368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:20.741413 18366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:12:20.742667 18353 server_base.cc:1034] running on GCE node
I20260811 19:12:20.743610 18353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:12:20.744879 18353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:12:20.746161 18353 hybrid_clock.cc:648] HybridClock initialized: now 1786475540746098 us; error 41 us; skew 500 ppm
I20260811 19:12:20.746654 18353 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:12:20 dist-test-slave-t7g5 krb5kdc[18187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475540, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:12:20.750433 18353 init.cc:377] Logged in from keytab as kudu/127.3.61.129@KRBTEST.COM (short username kudu)
I20260811 19:12:20.752182 18353 webserver.cc:460] Webserver started at http://127.3.61.129:32847/ using document root <none> and password file <none>
I20260811 19:12:20.752970 18353 fs_manager.cc:359] Metadata directory not provided
I20260811 19:12:20.753111 18353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:12:20.753449 18353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:12:20.757191 18353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-0/data/instance:
uuid: "01f89bb7c69e4bb99716d66c6bb1ceaf"
format_stamp: "Formatted at 2026-08-11 19:12:20 on dist-test-slave-t7g5"
server_key: "9b2fb55bb70a3c511fed6ec7b04c6c7e"
server_key_iv: "5fb17e57db2b3d18748082b402df68a3"
server_key_version: "encryptionkey@0"
I20260811 19:12:20.758139 18353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-0/wal/instance:
uuid: "01f89bb7c69e4bb99716d66c6bb1ceaf"
format_stamp: "Formatted at 2026-08-11 19:12:20 on dist-test-slave-t7g5"
server_key: "9b2fb55bb70a3c511fed6ec7b04c6c7e"
server_key_iv: "5fb17e57db2b3d18748082b402df68a3"
server_key_version: "encryptionkey@0"
I20260811 19:12:20.764825 18353 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.006s
I20260811 19:12:20.768729 18385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:20.770346 18353 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:12:20.770535 18353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-0/wal
uuid: "01f89bb7c69e4bb99716d66c6bb1ceaf"
format_stamp: "Formatted at 2026-08-11 19:12:20 on dist-test-slave-t7g5"
server_key: "9b2fb55bb70a3c511fed6ec7b04c6c7e"
server_key_iv: "5fb17e57db2b3d18748082b402df68a3"
server_key_version: "encryptionkey@0"
I20260811 19:12:20.770748 18353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:12:20.795073 18353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:12:20.801872 18353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:12:20.802218 18353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:12:20.803304 18353 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:12:20.805282 18353 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:12:20.805394 18353 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:20.805528 18353 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:12:20.805585 18353 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:20.843156 18353 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.129:42011
I20260811 19:12:20.843197 18570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.129:42011 every 8 connection(s)
I20260811 19:12:20.845036 18353 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
I20260811 19:12:20.847214 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 18353
I20260811 19:12:20.847414 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-0/wal/instance
I20260811 19:12:20.848187 3318 external_mini_cluster.cc:1428] Setting key b1059f719d20167b35c744ed9a664654
Aug 11 19:12:20 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475540, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.129@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:12:20.871412 18575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:20.847939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:37303 (local address 127.3.61.190:35751)
0811 19:12:20.848458 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:20.848465 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:20.849544 (+ 1079us) server_negotiation.cc:407] Connection header received
0811 19:12:20.851535 (+ 1991us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:20.851540 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:20.851625 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:20.851739 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:20.854256 (+ 2517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:20.855248 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:20.857337 (+ 2089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:20.857926 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:20.858078 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:20.862515 (+ 4437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:20.862524 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:20.862608 (+ 84us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:20.863485 (+ 877us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:20.867122 (+ 3637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:20.868344 (+ 1222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:20.868351 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:20.868354 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:20.868710 (+ 356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:20.869365 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:20.869369 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:20.869371 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:20.869775 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:20.869899 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:20.870924 (+ 1025us) server_negotiation.cc:299] Negotiation successful
0811 19:12:20.871136 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":251,"threads_started":1}
I20260811 19:12:20.872843 18577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:20.848503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35751 (local address 127.3.61.129:37303)
0811 19:12:20.849370 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:20.849429 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:20.851246 (+ 1817us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:20.852251 (+ 1005us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:20.852264 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:20.853050 (+ 786us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:20.853989 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:20.854026 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:20.855587 (+ 1561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:20.855593 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:20.857119 (+ 1526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:20.857141 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:20.858227 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:20.858232 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:20.858607 (+ 375us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:20.859553 (+ 946us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:20.859579 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:20.862116 (+ 2537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:12:20.867362 (+ 5246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:20.867373 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:20.867395 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:20.868080 (+ 685us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:20.868888 (+ 808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:20.868893 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:20.868896 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:20.869202 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:20.869912 (+ 710us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:12:20.869921 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:12:20.870429 (+ 508us) client_negotiation.cc:769] Sending connection context
0811 19:12:20.870752 (+ 323us) client_negotiation.cc:240] Negotiation successful
0811 19:12:20.871374 (+ 622us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":348,"threads_started":1}
I20260811 19:12:20.874656 18572 heartbeater.cc:347] Connected to a master server at 127.3.61.190:35751
I20260811 19:12:20.875005 18572 heartbeater.cc:464] Registering TS with master...
I20260811 19:12:20.875881 18572 heartbeater.cc:511] Master 127.3.61.190:35751 requested a full tablet report, sending...
I20260811 19:12:20.878427 18257 ts_manager.cc:194] Registered new tserver with Master: 01f89bb7c69e4bb99716d66c6bb1ceaf (127.3.61.129:42011)
I20260811 19:12:20.880079 18257 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.129@KRBTEST.COM'} at 127.3.61.129:37303
WARNING: no policy specified for kudu/127.3.61.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab.
Entry for principal kudu/127.3.61.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab.
I20260811 19:12:20.922536 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.130:0
--local_ip_for_outbound_sockets=127.3.61.130
--webserver_interface=127.3.61.130
--webserver_port=0
--tserver_master_addrs=127.3.61.190:35751
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab
--principal=kudu/127.3.61.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:34597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:12:21.298374 18583 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:12:21.298727 18583 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:12:21.298862 18583 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:12:21.306824 18583 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:12:21.306993 18583 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:12:21.307154 18583 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.130
I20260811 19:12:21.317987 18583 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:34597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab
--principal=kudu/127.3.61.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.61.130
--webserver_port=0
--tserver_master_addrs=127.3.61.190:35751
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.130
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:12:21.319576 18583 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:12:21.320983 18583 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:12:21.335991 18595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:21.336021 18599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:21.336829 18596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:12:21.337328 18583 server_base.cc:1034] running on GCE node
I20260811 19:12:21.338058 18583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:12:21.339107 18583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:12:21.340356 18583 hybrid_clock.cc:648] HybridClock initialized: now 1786475541340284 us; error 64 us; skew 500 ppm
I20260811 19:12:21.340728 18583 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:12:21 dist-test-slave-t7g5 krb5kdc[18187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475541, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:12:21.344226 18583 init.cc:377] Logged in from keytab as kudu/127.3.61.130@KRBTEST.COM (short username kudu)
I20260811 19:12:21.346128 18583 webserver.cc:460] Webserver started at http://127.3.61.130:43725/ using document root <none> and password file <none>
I20260811 19:12:21.346909 18583 fs_manager.cc:359] Metadata directory not provided
I20260811 19:12:21.347041 18583 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:12:21.347378 18583 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:12:21.350564 18583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-1/data/instance:
uuid: "06684f8c03484fd98ccaa196e9d555df"
format_stamp: "Formatted at 2026-08-11 19:12:21 on dist-test-slave-t7g5"
server_key: "a37f9f5d90f26d447bf3d51c97fe4018"
server_key_iv: "623ae549a493b867b1ee033ac9e7d906"
server_key_version: "encryptionkey@0"
I20260811 19:12:21.351395 18583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-1/wal/instance:
uuid: "06684f8c03484fd98ccaa196e9d555df"
format_stamp: "Formatted at 2026-08-11 19:12:21 on dist-test-slave-t7g5"
server_key: "a37f9f5d90f26d447bf3d51c97fe4018"
server_key_iv: "623ae549a493b867b1ee033ac9e7d906"
server_key_version: "encryptionkey@0"
I20260811 19:12:21.357025 18583 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 19:12:21.361419 18611 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:21.363212 18583 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 19:12:21.363417 18583 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-1/wal
uuid: "06684f8c03484fd98ccaa196e9d555df"
format_stamp: "Formatted at 2026-08-11 19:12:21 on dist-test-slave-t7g5"
server_key: "a37f9f5d90f26d447bf3d51c97fe4018"
server_key_iv: "623ae549a493b867b1ee033ac9e7d906"
server_key_version: "encryptionkey@0"
I20260811 19:12:21.363636 18583 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:12:21.417771 18583 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:12:21.422963 18583 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:12:21.423280 18583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:12:21.424357 18583 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:12:21.425891 18583 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:12:21.425990 18583 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:21.426116 18583 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:12:21.426172 18583 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:21.465308 18583 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.130:46301
I20260811 19:12:21.465341 18778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.130:46301 every 8 connection(s)
I20260811 19:12:21.466771 18583 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
I20260811 19:12:21.469900 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 18583
I20260811 19:12:21.470088 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-1/wal/instance
I20260811 19:12:21.470592 3318 external_mini_cluster.cc:1428] Setting key 8955b577bad8476e51d9ff36bdd46a32
Aug 11 19:12:21 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475541, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.130@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:12:21.491976 18786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:21.470666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:35087 (local address 127.3.61.190:35751)
0811 19:12:21.471206 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:21.471213 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:21.472526 (+ 1313us) server_negotiation.cc:407] Connection header received
0811 19:12:21.474723 (+ 2197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:21.474728 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:21.474825 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:21.474923 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:21.476648 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:21.477522 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:21.479243 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:21.479690 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:21.479815 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:21.484209 (+ 4394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:21.484217 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:21.484221 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:21.484266 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:21.487555 (+ 3289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:21.488912 (+ 1357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:21.488918 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:21.488921 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:21.489159 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:21.489958 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:21.489963 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:21.489966 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:21.490330 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:21.490479 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:21.491474 (+ 995us) server_negotiation.cc:299] Negotiation successful
0811 19:12:21.491678 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":263,"threads_started":1}
I20260811 19:12:21.493489 18787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:21.471395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35751 (local address 127.3.61.130:35087)
0811 19:12:21.472337 (+ 942us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:21.472407 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:21.474404 (+ 1997us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:21.475198 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:21.475208 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:21.475831 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:21.476464 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:21.476483 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:21.477721 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:21.477725 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:21.479094 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:21.479109 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:21.479929 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:21.479937 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:21.480170 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:21.480959 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:21.480987 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:21.483593 (+ 2606us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:12:21.487802 (+ 4209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:21.487815 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:21.487841 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:21.488679 (+ 838us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:21.489318 (+ 639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:21.489324 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:21.489328 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:21.489634 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:21.490499 (+ 865us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:12:21.490509 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:12:21.491002 (+ 493us) client_negotiation.cc:769] Sending connection context
0811 19:12:21.491322 (+ 320us) client_negotiation.cc:240] Negotiation successful
0811 19:12:21.491934 (+ 612us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"thread_start_us":345,"threads_started":1}
I20260811 19:12:21.495321 18779 heartbeater.cc:347] Connected to a master server at 127.3.61.190:35751
I20260811 19:12:21.495702 18779 heartbeater.cc:464] Registering TS with master...
I20260811 19:12:21.496511 18779 heartbeater.cc:511] Master 127.3.61.190:35751 requested a full tablet report, sending...
I20260811 19:12:21.498225 18257 ts_manager.cc:194] Registered new tserver with Master: 06684f8c03484fd98ccaa196e9d555df (127.3.61.130:46301)
I20260811 19:12:21.499087 18257 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.130@KRBTEST.COM'} at 127.3.61.130:35087
WARNING: no policy specified for kudu/127.3.61.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab.
Entry for principal kudu/127.3.61.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab.
I20260811 19:12:21.546123 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.131:0
--local_ip_for_outbound_sockets=127.3.61.131
--webserver_interface=127.3.61.131
--webserver_port=0
--tserver_master_addrs=127.3.61.190:35751
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab
--principal=kudu/127.3.61.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:34597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:12:21.884832 18572 heartbeater.cc:503] Master 127.3.61.190:35751 was elected leader, sending a full tablet report...
W20260811 19:12:21.930941 18790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:12:21.931270 18790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:12:21.931403 18790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:12:21.939039 18790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:12:21.939313 18790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:12:21.939522 18790 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.131
I20260811 19:12:21.950560 18790 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:34597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab
--principal=kudu/127.3.61.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.61.131
--webserver_port=0
--tserver_master_addrs=127.3.61.190:35751
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.131
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:12:21.952016 18790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:12:21.953379 18790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:12:21.967461 18796 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:21.967504 18797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:21.967533 18802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:12:21.968981 18790 server_base.cc:1034] running on GCE node
I20260811 19:12:21.969692 18790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:12:21.970995 18790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:12:21.972348 18790 hybrid_clock.cc:648] HybridClock initialized: now 1786475541972283 us; error 69 us; skew 500 ppm
I20260811 19:12:21.972748 18790 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:12:21 dist-test-slave-t7g5 krb5kdc[18187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475541, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:12:21.976480 18790 init.cc:377] Logged in from keytab as kudu/127.3.61.131@KRBTEST.COM (short username kudu)
I20260811 19:12:21.978345 18790 webserver.cc:460] Webserver started at http://127.3.61.131:39215/ using document root <none> and password file <none>
I20260811 19:12:21.979127 18790 fs_manager.cc:359] Metadata directory not provided
I20260811 19:12:21.979257 18790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:12:21.979583 18790 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:12:21.982813 18790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-2/data/instance:
uuid: "a25ce9f5345b40109cbc777062b032e5"
format_stamp: "Formatted at 2026-08-11 19:12:21 on dist-test-slave-t7g5"
server_key: "de291d3533e4607fb6cae1eaaf60b605"
server_key_iv: "fe6aeb1f1ac185ddadca4eafa289b43a"
server_key_version: "encryptionkey@0"
I20260811 19:12:21.983677 18790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-2/wal/instance:
uuid: "a25ce9f5345b40109cbc777062b032e5"
format_stamp: "Formatted at 2026-08-11 19:12:21 on dist-test-slave-t7g5"
server_key: "de291d3533e4607fb6cae1eaaf60b605"
server_key_iv: "fe6aeb1f1ac185ddadca4eafa289b43a"
server_key_version: "encryptionkey@0"
I20260811 19:12:21.989773 18790 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260811 19:12:21.993891 18816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:21.995541 18790 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 19:12:21.995733 18790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-2/wal
uuid: "a25ce9f5345b40109cbc777062b032e5"
format_stamp: "Formatted at 2026-08-11 19:12:21 on dist-test-slave-t7g5"
server_key: "de291d3533e4607fb6cae1eaaf60b605"
server_key_iv: "fe6aeb1f1ac185ddadca4eafa289b43a"
server_key_version: "encryptionkey@0"
I20260811 19:12:21.995954 18790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:12:22.025410 18790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:12:22.031661 18790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:12:22.032094 18790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:12:22.033536 18790 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:12:22.035569 18790 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:12:22.035681 18790 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:22.035815 18790 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:12:22.035878 18790 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:22.074754 18790 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.131:44907
I20260811 19:12:22.074767 19005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.131:44907 every 8 connection(s)
I20260811 19:12:22.076400 18790 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
I20260811 19:12:22.081925 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 18790
I20260811 19:12:22.082136 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-2/wal/instance
I20260811 19:12:22.082715 3318 external_mini_cluster.cc:1428] Setting key f403371f19ce4a559ce0cbc0854a9c2f
Aug 11 19:12:22 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475541, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.131@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:12:22.102290 19010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:22.079535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:43735 (local address 127.3.61.190:35751)
0811 19:12:22.080047 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:22.080051 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:22.081139 (+ 1088us) server_negotiation.cc:407] Connection header received
0811 19:12:22.083837 (+ 2698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:22.083841 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:22.083968 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:22.084114 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:22.086467 (+ 2353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:22.087355 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:22.089541 (+ 2186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:22.090054 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:22.090172 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:22.094505 (+ 4333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:22.094511 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:22.094515 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:22.094555 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:22.097678 (+ 3123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:22.099139 (+ 1461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:22.099143 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:22.099146 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:22.099366 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:22.100006 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:22.100010 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:22.100012 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:22.100340 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:22.100469 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:22.101780 (+ 1311us) server_negotiation.cc:299] Negotiation successful
0811 19:12:22.101998 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":250,"threads_started":1}
I20260811 19:12:22.103737 19011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:22.080086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35751 (local address 127.3.61.131:43735)
0811 19:12:22.080954 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:22.080999 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:22.083483 (+ 2484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:22.084566 (+ 1083us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:22.084576 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:22.085339 (+ 763us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:22.086257 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:22.086281 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:22.087515 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:22.087520 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:22.089296 (+ 1776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:22.089326 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:22.090369 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:22.090376 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:22.090623 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:22.091375 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:22.091404 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:22.094208 (+ 2804us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:12:22.097851 (+ 3643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:22.097861 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:22.097881 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:22.098925 (+ 1044us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:22.099562 (+ 637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:22.099569 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:22.099573 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:22.099857 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:22.100629 (+ 772us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:12:22.100640 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:12:22.101202 (+ 562us) client_negotiation.cc:769] Sending connection context
0811 19:12:22.101551 (+ 349us) client_negotiation.cc:240] Negotiation successful
0811 19:12:22.102239 (+ 688us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":390,"threads_started":1}
I20260811 19:12:22.105924 19006 heartbeater.cc:347] Connected to a master server at 127.3.61.190:35751
I20260811 19:12:22.106381 19006 heartbeater.cc:464] Registering TS with master...
I20260811 19:12:22.107259 19006 heartbeater.cc:511] Master 127.3.61.190:35751 requested a full tablet report, sending...
I20260811 19:12:22.109499 18257 ts_manager.cc:194] Registered new tserver with Master: a25ce9f5345b40109cbc777062b032e5 (127.3.61.131:44907)
I20260811 19:12:22.110548 18257 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.131@KRBTEST.COM'} at 127.3.61.131:43735
I20260811 19:12:22.112805 3318 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.3.61.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.61.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.61.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/krb5kdc/oryx_127.3.61.129.keytab.
Entry for principal oryx/127.3.61.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/krb5kdc/oryx_127.3.61.129.keytab.
WARNING: no policy specified for oryx/127.3.61.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.61.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.61.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/krb5kdc/oryx_127.3.61.130.keytab.
Entry for principal oryx/127.3.61.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/krb5kdc/oryx_127.3.61.130.keytab.
WARNING: no policy specified for oryx/127.3.61.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.61.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.61.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/krb5kdc/oryx_127.3.61.131.keytab.
Entry for principal oryx/127.3.61.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/krb5kdc/oryx_127.3.61.131.keytab.
I20260811 19:12:22.325795 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 18353
Aug 11 19:12:22 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475539, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.129@KRBTEST.COM
I20260811 19:12:22.346531 19023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:22.327762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45900 (local address 127.3.61.129:42011)
0811 19:12:22.328649 (+ 887us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:22.328659 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:22.328694 (+ 35us) server_negotiation.cc:407] Connection header received
0811 19:12:22.328791 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:22.328799 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:22.329080 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:22.329312 (+ 232us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:22.331057 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:22.332305 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:22.333727 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:22.334347 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:22.334464 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:22.339119 (+ 4655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:22.339136 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:22.339154 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:22.339197 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:22.342398 (+ 3201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:22.343619 (+ 1221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:22.343632 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:22.343634 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:22.343809 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:22.344358 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:22.344363 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:22.344365 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:22.344787 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:22.345057 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:22.345721 (+ 664us) server_negotiation.cc:299] Negotiation successful
0811 19:12:22.346066 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":684,"thread_start_us":469,"threads_started":1}
I20260811 19:12:22.347977 18565 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:12:22.438277 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 18353
I20260811 19:12:22.470629 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 18583
Aug 11 19:12:22 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475539, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.130@KRBTEST.COM
I20260811 19:12:22.490186 19027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:22.471634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48322 (local address 127.3.61.130:46301)
0811 19:12:22.472198 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:22.472206 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:22.472232 (+ 26us) server_negotiation.cc:407] Connection header received
0811 19:12:22.472301 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:22.472307 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:22.472526 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:22.472761 (+ 235us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:22.474074 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:22.475107 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:22.476387 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:22.476982 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:22.477137 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:22.481837 (+ 4700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:22.481860 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:22.481884 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:22.481943 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:22.486109 (+ 4166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:22.487434 (+ 1325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:22.487445 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:22.487448 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:22.487685 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:22.488232 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:22.488237 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:22.488239 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:22.488701 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:22.488990 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:22.489538 (+ 548us) server_negotiation.cc:299] Negotiation successful
0811 19:12:22.489858 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":259,"threads_started":1}
I20260811 19:12:22.491551 18773 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:12:22.579762 18779 heartbeater.cc:503] Master 127.3.61.190:35751 was elected leader, sending a full tablet report...
I20260811 19:12:22.581809 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 18583
W20260811 19:12:22.607645 18237 connection.cc:535] server connection from 127.3.61.130:35087 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 19:12:22.608134 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 18790
Aug 11 19:12:22 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475539, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.131@KRBTEST.COM
I20260811 19:12:22.627465 19029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:22.609325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53948 (local address 127.3.61.131:44907)
0811 19:12:22.610059 (+ 734us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:22.610070 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:22.610101 (+ 31us) server_negotiation.cc:407] Connection header received
0811 19:12:22.610168 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:22.610175 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:22.610405 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:22.610567 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:22.611978 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:22.613009 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:22.614480 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:22.615066 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:22.615188 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:22.620331 (+ 5143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:22.620347 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:22.620366 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:22.620414 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:22.624155 (+ 3741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:22.624947 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:22.624955 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:22.624958 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:22.625111 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:22.625543 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:22.625547 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:22.625549 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:22.626036 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:22.626312 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:22.626763 (+ 451us) server_negotiation.cc:299] Negotiation successful
0811 19:12:22.627042 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":535,"thread_start_us":367,"threads_started":1}
I20260811 19:12:22.628850 18999 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:12:22.717137 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 18790
I20260811 19:12:22.744361 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 18208
I20260811 19:12:22.746007 18301 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:12:22.856582 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 18208
I20260811 19:12:22.892900 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:35751
--webserver_interface=127.3.61.190
--webserver_port=43267
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:34597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.61.190:35751
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:12:23.287497 19035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:12:23.287992 19035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:12:23.288311 19035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:12:23.297432 19035 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:12:23.297602 19035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:12:23.297688 19035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:12:23.297796 19035 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:12:23.297866 19035 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:12:23.309552 19035 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:34597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.61.190:35751
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:35751
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.61.190
--webserver_port=43267
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:12:23.311064 19035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:12:23.312518 19035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:12:23.325454 19044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:23.325984 19050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:23.326059 19046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:12:23.327121 19035 server_base.cc:1034] running on GCE node
I20260811 19:12:23.328109 19035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:12:23.330201 19035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:12:23.331545 19035 hybrid_clock.cc:648] HybridClock initialized: now 1786475543331460 us; error 38 us; skew 500 ppm
I20260811 19:12:23.332155 19035 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:12:23 dist-test-slave-t7g5 krb5kdc[18187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475543, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:12:23.336167 19035 init.cc:377] Logged in from keytab as kudu/127.3.61.190@KRBTEST.COM (short username kudu)
I20260811 19:12:23.337947 19035 webserver.cc:460] Webserver started at http://127.3.61.190:43267/ using document root <none> and password file <none>
I20260811 19:12:23.338718 19035 fs_manager.cc:359] Metadata directory not provided
I20260811 19:12:23.338876 19035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:12:23.345917 19035 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.003s
I20260811 19:12:23.349594 19059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:23.351851 19035 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 19:12:23.352092 19035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/master-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/master-0/wal
uuid: "45cde14fb6a94f3d8f9c9031f823cc19"
format_stamp: "Formatted at 2026-08-11 19:12:20 on dist-test-slave-t7g5"
server_key: "ffd185c0a338a9d0624af287350d5936"
server_key_iv: "0427ae1d2c2c75996a6406c159b8cec0"
server_key_version: "encryptionkey@0"
I20260811 19:12:23.353211 19035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:12:23.365684 19035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:12:23.371559 19035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:12:23.371855 19035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:12:23.392983 19035 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.190:35751
I20260811 19:12:23.393019 19140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.190:35751 every 8 connection(s)
I20260811 19:12:23.394762 19035 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
I20260811 19:12:23.398379 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 19035
I20260811 19:12:23.399367 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.129:42011
--local_ip_for_outbound_sockets=127.3.61.129
--tserver_master_addrs=127.3.61.190:35751
--webserver_port=32847
--webserver_interface=127.3.61.129
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab
--principal=kudu/127.3.61.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:34597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.3.61.129
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/krb5kdc/oryx_127.3.61.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:12:23.403501 19142 sys_catalog.cc:263] Verifying existing consensus state
I20260811 19:12:23.408102 19142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45cde14fb6a94f3d8f9c9031f823cc19: Bootstrap starting.
I20260811 19:12:23.429986 19142 log.cc:824] T 00000000000000000000000000000000 P 45cde14fb6a94f3d8f9c9031f823cc19: Log is configured to *not* fsync() on all Append() calls
I20260811 19:12:23.440289 19142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45cde14fb6a94f3d8f9c9031f823cc19: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 19:12:23.440896 19142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45cde14fb6a94f3d8f9c9031f823cc19: Bootstrap complete.
I20260811 19:12:23.450369 19142 raft_consensus.cc:348] T 00000000000000000000000000000000 P 45cde14fb6a94f3d8f9c9031f823cc19 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45cde14fb6a94f3d8f9c9031f823cc19" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35751 } }
I20260811 19:12:23.451308 19142 raft_consensus.cc:729] T 00000000000000000000000000000000 P 45cde14fb6a94f3d8f9c9031f823cc19 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45cde14fb6a94f3d8f9c9031f823cc19, State: Initialized, Role: FOLLOWER
I20260811 19:12:23.452028 19142 consensus_queue.cc:260] T 00000000000000000000000000000000 P 45cde14fb6a94f3d8f9c9031f823cc19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45cde14fb6a94f3d8f9c9031f823cc19" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35751 } }
I20260811 19:12:23.452270 19142 raft_consensus.cc:388] T 00000000000000000000000000000000 P 45cde14fb6a94f3d8f9c9031f823cc19 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:12:23.452390 19142 raft_consensus.cc:482] T 00000000000000000000000000000000 P 45cde14fb6a94f3d8f9c9031f823cc19 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:12:23.452571 19142 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 45cde14fb6a94f3d8f9c9031f823cc19 [term 1 FOLLOWER]: Advancing to term 2
I20260811 19:12:23.456838 19142 raft_consensus.cc:504] T 00000000000000000000000000000000 P 45cde14fb6a94f3d8f9c9031f823cc19 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45cde14fb6a94f3d8f9c9031f823cc19" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35751 } }
I20260811 19:12:23.457425 19142 leader_election.cc:302] T 00000000000000000000000000000000 P 45cde14fb6a94f3d8f9c9031f823cc19 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 45cde14fb6a94f3d8f9c9031f823cc19; no voters:
I20260811 19:12:23.458135 19142 leader_election.cc:288] T 00000000000000000000000000000000 P 45cde14fb6a94f3d8f9c9031f823cc19 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 19:12:23.458340 19150 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 45cde14fb6a94f3d8f9c9031f823cc19 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 19:12:23.460659 19142 sys_catalog.cc:564] T 00000000000000000000000000000000 P 45cde14fb6a94f3d8f9c9031f823cc19 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:12:23.465981 19150 raft_consensus.cc:686] T 00000000000000000000000000000000 P 45cde14fb6a94f3d8f9c9031f823cc19 [term 2 LEADER]: Becoming Leader. State: Replica: 45cde14fb6a94f3d8f9c9031f823cc19, State: Running, Role: LEADER
I20260811 19:12:23.466770 19150 consensus_queue.cc:237] T 00000000000000000000000000000000 P 45cde14fb6a94f3d8f9c9031f823cc19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45cde14fb6a94f3d8f9c9031f823cc19" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35751 } }
I20260811 19:12:23.469918 19151 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45cde14fb6a94f3d8f9c9031f823cc19 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "45cde14fb6a94f3d8f9c9031f823cc19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45cde14fb6a94f3d8f9c9031f823cc19" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35751 } } }
I20260811 19:12:23.470431 19151 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45cde14fb6a94f3d8f9c9031f823cc19 [sys.catalog]: This master's current role is: LEADER
I20260811 19:12:23.471112 19152 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45cde14fb6a94f3d8f9c9031f823cc19 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 45cde14fb6a94f3d8f9c9031f823cc19. Latest consensus state: current_term: 2 leader_uuid: "45cde14fb6a94f3d8f9c9031f823cc19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45cde14fb6a94f3d8f9c9031f823cc19" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35751 } } }
I20260811 19:12:23.471412 19152 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45cde14fb6a94f3d8f9c9031f823cc19 [sys.catalog]: This master's current role is: LEADER
I20260811 19:12:23.473766 19162 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:12:23.480039 19162 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:12:23.481726 19162 catalog_manager.cc:1259] Loaded cluster ID: 64396ceafe71488baa6c058b98b25268
I20260811 19:12:23.481927 19162 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:12:23.487344 19162 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:12:23.489305 19162 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 45cde14fb6a94f3d8f9c9031f823cc19: Loaded TSK: 0
I20260811 19:12:23.490577 19162 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 19:12:23.820812 19145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:12:23.821169 19145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:12:23.821301 19145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:12:23.829258 19145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:12:23.829407 19145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:12:23.829569 19145 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.129
I20260811 19:12:23.839819 19145 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:34597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/krb5kdc/oryx_127.3.61.129.keytab
--principal=oryx/127.3.61.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.129:42011
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.61.129
--webserver_port=32847
--tserver_master_addrs=127.3.61.190:35751
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.129
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:12:23.841967 19145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:12:23.843717 19145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:12:23.859345 19195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:23.859360 19196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:23.862960 19198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:12:23.864250 19145 server_base.cc:1034] running on GCE node
I20260811 19:12:23.865033 19145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:12:23.866722 19145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:12:23.868073 19145 hybrid_clock.cc:648] HybridClock initialized: now 1786475543867996 us; error 65 us; skew 500 ppm
I20260811 19:12:23.868506 19145 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:12:23 dist-test-slave-t7g5 krb5kdc[18187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475543, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.61.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:12:23.872134 19145 init.cc:377] Logged in from keytab as oryx/127.3.61.129@KRBTEST.COM (short username oryx)
I20260811 19:12:23.874109 19145 webserver.cc:460] Webserver started at http://127.3.61.129:32847/ using document root <none> and password file <none>
I20260811 19:12:23.874912 19145 fs_manager.cc:359] Metadata directory not provided
I20260811 19:12:23.875061 19145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:12:23.880945 19145 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 19:12:23.884069 19210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:23.885763 19145 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:12:23.885964 19145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-0/wal
uuid: "01f89bb7c69e4bb99716d66c6bb1ceaf"
format_stamp: "Formatted at 2026-08-11 19:12:20 on dist-test-slave-t7g5"
server_key: "9b2fb55bb70a3c511fed6ec7b04c6c7e"
server_key_iv: "5fb17e57db2b3d18748082b402df68a3"
server_key_version: "encryptionkey@0"
I20260811 19:12:23.887223 19145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:12:23.908404 19145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:12:23.913280 19145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:12:23.913594 19145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:12:23.914726 19145 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:12:23.916750 19145 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:12:23.916867 19145 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:23.917007 19145 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:12:23.917081 19145 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:23.955211 19145 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.129:42011
I20260811 19:12:23.955271 19383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.129:42011 every 8 connection(s)
I20260811 19:12:23.957001 19145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
I20260811 19:12:23.964766 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 19145
I20260811 19:12:23.965560 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.130:46301
--local_ip_for_outbound_sockets=127.3.61.130
--tserver_master_addrs=127.3.61.190:35751
--webserver_port=43725
--webserver_interface=127.3.61.130
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab
--principal=kudu/127.3.61.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:34597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.3.61.130
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/krb5kdc/oryx_127.3.61.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 19:12:23 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.129@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:12:23 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.129@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:12:23.979388 19389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:23.960756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35751 (local address 127.3.61.129:43333)
0811 19:12:23.961795 (+ 1039us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:23.961847 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:23.963658 (+ 1811us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:23.965465 (+ 1807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:23.965473 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:23.966135 (+ 662us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:23.967062 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:23.967096 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:23.969576 (+ 2480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:23.969582 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:23.970991 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:23.971009 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:23.972420 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:23.972426 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:23.972741 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:23.973591 (+ 850us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:23.973626 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:23.978266 (+ 4640us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.61.190:35751: Server oryx/127.3.61.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":692,"thread_start_us":507,"threads_started":1}
W20260811 19:12:23.980367 19386 heartbeater.cc:644] Failed to heartbeat to 127.3.61.190:35751 (0 consecutive failures): Not authorized: Failed to ping master at 127.3.61.190:35751: Client connection negotiation failed: client connection to 127.3.61.190:35751: Server oryx/127.3.61.190@KRBTEST.COM not found in Kerberos database
W20260811 19:12:23.982002 19388 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:12:23.960630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:43333 (local address 127.3.61.190:35751)
0811 19:12:23.961562 (+ 932us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:23.961579 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:23.962015 (+ 436us) server_negotiation.cc:407] Connection header received
0811 19:12:23.964329 (+ 2314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:23.964350 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:23.964848 (+ 498us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:23.965296 (+ 448us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:23.967423 (+ 2127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:23.969315 (+ 1892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:23.971177 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:23.972127 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:23.972333 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:23.979240 (+ 6907us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.61.129:43333: BlockingRecv error: recv got EOF from 127.3.61.129:43333 (error 108)
Metrics: {"server-negotiator.queue_time_us":628,"thread_start_us":354,"threads_started":1}
Aug 11 19:12:23 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.129@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:12:23 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.129@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:12:23.990804 19389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:23.981156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35751 (local address 127.3.61.129:41129)
0811 19:12:23.981466 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:23.981489 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:23.981623 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:23.982430 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:23.982435 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:23.982820 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:23.982972 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:23.982980 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:23.984045 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:23.984049 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:23.984959 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:23.984971 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:23.985749 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:23.985756 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:23.985936 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:23.986679 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:23.986710 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:23.990492 (+ 3782us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.61.190:35751: Server oryx/127.3.61.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186}
W20260811 19:12:23.991526 19394 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:12:23.981304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:41129 (local address 127.3.61.190:35751)
0811 19:12:23.982094 (+ 790us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:23.982100 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:23.982123 (+ 23us) server_negotiation.cc:407] Connection header received
0811 19:12:23.982190 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:23.982195 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:23.982271 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:23.982379 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:23.983118 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:23.983901 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:23.985101 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:23.985582 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:23.985729 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:23.990781 (+ 5052us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.61.129:41129: BlockingRecv error: recv got EOF from 127.3.61.129:41129 (error 108)
Metrics: {"server-negotiator.queue_time_us":687,"thread_start_us":571,"threads_started":1}
Aug 11 19:12:23 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.129@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:12:23 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.129@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:12:24.002487 19389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:23.991655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35751 (local address 127.3.61.129:55633)
0811 19:12:23.991853 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:23.991873 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:23.991988 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:23.992990 (+ 1002us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:23.992995 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:23.993411 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:23.993619 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:23.993630 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:23.994795 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:23.994799 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:23.995981 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:23.996003 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:23.996886 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:23.996892 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:23.997099 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:23.998093 (+ 994us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:23.998116 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:24.002097 (+ 3981us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.61.190:35751: Server oryx/127.3.61.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":57}
W20260811 19:12:24.002715 19394 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:12:23.992299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:55633 (local address 127.3.61.190:35751)
0811 19:12:23.992545 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:23.992552 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:23.992581 (+ 29us) server_negotiation.cc:407] Connection header received
0811 19:12:23.992673 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:23.992677 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:23.992783 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:23.992908 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:23.993835 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:23.994633 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:23.996226 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:23.996699 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:23.996831 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:24.002386 (+ 5555us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.61.129:55633: BlockingRecv error: recv got EOF from 127.3.61.129:55633 (error 108)
Metrics: {"server-negotiator.queue_time_us":103}
W20260811 19:12:24.003122 19386 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 19:12:24.365761 19390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:12:24.366150 19390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:12:24.366289 19390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:12:24.374931 19390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:12:24.375079 19390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:12:24.375252 19390 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.130
I20260811 19:12:24.385978 19390 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:34597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/krb5kdc/oryx_127.3.61.130.keytab
--principal=oryx/127.3.61.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.130:46301
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.61.130
--webserver_port=43725
--tserver_master_addrs=127.3.61.190:35751
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.130
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:12:24.387501 19390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:12:24.389106 19390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:12:24.406363 19400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:24.406385 19401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:24.407503 19403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:12:24.408053 19390 server_base.cc:1034] running on GCE node
I20260811 19:12:24.408776 19390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:12:24.409889 19390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:12:24.412086 19390 hybrid_clock.cc:648] HybridClock initialized: now 1786475544411974 us; error 57 us; skew 500 ppm
I20260811 19:12:24.412482 19390 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:12:24 dist-test-slave-t7g5 krb5kdc[18187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475544, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.61.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:12:24.416173 19390 init.cc:377] Logged in from keytab as oryx/127.3.61.130@KRBTEST.COM (short username oryx)
I20260811 19:12:24.418118 19390 webserver.cc:460] Webserver started at http://127.3.61.130:43725/ using document root <none> and password file <none>
I20260811 19:12:24.418864 19390 fs_manager.cc:359] Metadata directory not provided
I20260811 19:12:24.419003 19390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:12:24.426884 19390 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 19:12:24.430259 19414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:24.431917 19390 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 19:12:24.432130 19390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-1/wal
uuid: "06684f8c03484fd98ccaa196e9d555df"
format_stamp: "Formatted at 2026-08-11 19:12:21 on dist-test-slave-t7g5"
server_key: "a37f9f5d90f26d447bf3d51c97fe4018"
server_key_iv: "623ae549a493b867b1ee033ac9e7d906"
server_key_version: "encryptionkey@0"
I20260811 19:12:24.433207 19390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:12:24.446295 19390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:12:24.451700 19390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:12:24.452039 19390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:12:24.453266 19390 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:12:24.454779 19390 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:12:24.454885 19390 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:24.454963 19390 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:12:24.455024 19390 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:24.492761 19390 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.130:46301
I20260811 19:12:24.492827 19595 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.130:46301 every 8 connection(s)
I20260811 19:12:24.494374 19390 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
I20260811 19:12:24.504899 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 19390
I20260811 19:12:24.505563 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.131:44907
--local_ip_for_outbound_sockets=127.3.61.131
--tserver_master_addrs=127.3.61.190:35751
--webserver_port=39215
--webserver_interface=127.3.61.131
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab
--principal=kudu/127.3.61.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:34597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.3.61.131
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/krb5kdc/oryx_127.3.61.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 19:12:24 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.130@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:12:24 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.130@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 19:12:24.514048 19394 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:12:24.498036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:52017 (local address 127.3.61.190:35751)
0811 19:12:24.498310 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:24.498317 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:24.499934 (+ 1617us) server_negotiation.cc:407] Connection header received
0811 19:12:24.501911 (+ 1977us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:24.501917 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:24.502103 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:24.502236 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:24.504080 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:24.505032 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:24.507483 (+ 2451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:24.507991 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:24.508126 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:24.513780 (+ 5654us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.61.130:52017: BlockingRecv error: recv got EOF from 127.3.61.130:52017 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
I20260811 19:12:24.514786 19599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:24.498500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35751 (local address 127.3.61.130:52017)
0811 19:12:24.499695 (+ 1195us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:24.499740 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:24.501541 (+ 1801us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:24.502536 (+ 995us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:24.502545 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:24.503143 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:24.503807 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:24.503827 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:24.505383 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:24.505392 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:24.507228 (+ 1836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:24.507252 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:24.508285 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:24.508295 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:24.508578 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:24.509343 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:24.509380 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:24.513444 (+ 4064us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.61.190:35751: Server oryx/127.3.61.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":677,"thread_start_us":361,"threads_started":1}
W20260811 19:12:24.515655 19596 heartbeater.cc:644] Failed to heartbeat to 127.3.61.190:35751 (0 consecutive failures): Not authorized: Failed to ping master at 127.3.61.190:35751: Client connection negotiation failed: client connection to 127.3.61.190:35751: Server oryx/127.3.61.190@KRBTEST.COM not found in Kerberos database
Aug 11 19:12:24 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.130@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:12:24 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.130@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:12:24.525868 19599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:24.516291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35751 (local address 127.3.61.130:44683)
0811 19:12:24.516490 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:24.516509 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:24.516609 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:24.516938 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:24.516942 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:24.517287 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:24.517458 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:24.517466 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:24.518670 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:24.518675 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:24.519953 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:24.519969 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:24.520760 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:24.520766 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:24.520938 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:24.521718 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:24.521740 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:24.525543 (+ 3803us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.61.190:35751: Server oryx/127.3.61.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":56}
W20260811 19:12:24.526615 19394 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:12:24.516431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:44683 (local address 127.3.61.190:35751)
0811 19:12:24.516596 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:24.516602 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:24.516625 (+ 23us) server_negotiation.cc:407] Connection header received
0811 19:12:24.516693 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:24.516698 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:24.516773 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:24.516872 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:24.517608 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:24.518514 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:24.520137 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:24.520594 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:24.520713 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:24.525937 (+ 5224us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.61.130:44683: BlockingRecv error: recv got EOF from 127.3.61.130:44683 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 11 19:12:24 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.130@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:12:24 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.130@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:12:24.537077 19599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:24.526797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35751 (local address 127.3.61.130:34725)
0811 19:12:24.526991 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:24.527011 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:24.527128 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:24.527581 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:24.527585 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:24.527879 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:24.528050 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:24.528058 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:24.529601 (+ 1543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:24.529605 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:24.530610 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:24.530624 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:24.531730 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:24.531747 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:24.532014 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:24.532880 (+ 866us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:24.532904 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:24.536783 (+ 3879us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.61.190:35751: Server oryx/127.3.61.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":63}
W20260811 19:12:24.537436 19394 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:12:24.527050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:34725 (local address 127.3.61.190:35751)
0811 19:12:24.527245 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:24.527251 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:24.527269 (+ 18us) server_negotiation.cc:407] Connection header received
0811 19:12:24.527339 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:24.527343 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:24.527414 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:24.527516 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:24.528202 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:24.529415 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:24.530767 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:24.531379 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:24.531504 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:24.537145 (+ 5641us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.61.130:34725: BlockingRecv error: recv got EOF from 127.3.61.130:34725 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260811 19:12:24.537442 19596 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 19:12:24.909191 19600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:12:24.909556 19600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:12:24.909749 19600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:12:24.917498 19600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:12:24.917696 19600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:12:24.917855 19600 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.131
I20260811 19:12:24.928871 19600 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:34597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/krb5kdc/oryx_127.3.61.131.keytab
--principal=oryx/127.3.61.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.131:44907
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.61.131
--webserver_port=39215
--tserver_master_addrs=127.3.61.190:35751
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.131
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:12:24.930583 19600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:12:24.932103 19600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:12:24.947820 19616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:24.947908 19617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:24.947880 19619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:12:24.948874 19600 server_base.cc:1034] running on GCE node
I20260811 19:12:24.949777 19600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:12:24.950994 19600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:12:24.952266 19600 hybrid_clock.cc:648] HybridClock initialized: now 1786475544952210 us; error 51 us; skew 500 ppm
I20260811 19:12:24.952764 19600 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:12:24 dist-test-slave-t7g5 krb5kdc[18187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475544, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.61.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:12:24.956452 19600 init.cc:377] Logged in from keytab as oryx/127.3.61.131@KRBTEST.COM (short username oryx)
I20260811 19:12:24.958592 19600 webserver.cc:460] Webserver started at http://127.3.61.131:39215/ using document root <none> and password file <none>
I20260811 19:12:24.959693 19600 fs_manager.cc:359] Metadata directory not provided
I20260811 19:12:24.959861 19600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:12:24.968562 19600 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 19:12:24.972719 19630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:24.974942 19600 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260811 19:12:24.975200 19600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-2/wal
uuid: "a25ce9f5345b40109cbc777062b032e5"
format_stamp: "Formatted at 2026-08-11 19:12:21 on dist-test-slave-t7g5"
server_key: "de291d3533e4607fb6cae1eaaf60b605"
server_key_iv: "fe6aeb1f1ac185ddadca4eafa289b43a"
server_key_version: "encryptionkey@0"
I20260811 19:12:24.976428 19600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 11 19:12:25 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.129@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:12:25.015156 19600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
Aug 11 19:12:25 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.129@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:12:25.017040 19635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:25.003884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35751 (local address 127.3.61.129:53797)
0811 19:12:25.004508 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:25.004530 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:25.004709 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:25.005054 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:25.005059 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:25.006105 (+ 1046us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:25.006491 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:25.006513 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:25.007912 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:25.007920 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:25.009784 (+ 1864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:25.009820 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:25.011328 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:25.011338 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:25.011566 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:25.012511 (+ 945us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:25.012537 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:25.016661 (+ 4124us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.61.190:35751: Server oryx/127.3.61.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":332,"threads_started":1}
W20260811 19:12:25.017436 19394 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:12:25.004101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:53797 (local address 127.3.61.190:35751)
0811 19:12:25.004322 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:25.004329 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:25.004673 (+ 344us) server_negotiation.cc:407] Connection header received
0811 19:12:25.004837 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:25.004841 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:25.004913 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:25.005008 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:25.006834 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:25.007639 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:25.010160 (+ 2521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:25.010978 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:25.011144 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:25.016961 (+ 5817us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.61.129:53797: BlockingRecv error: recv got EOF from 127.3.61.129:53797 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 19:12:25.021145 19600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:12:25.021481 19600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:12:25.022742 19600 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:12:25.024466 19600 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:12:25.024580 19600 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:25.024693 19600 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:12:25.024768 19600 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:25.068789 19600 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.131:44907
I20260811 19:12:25.068863 19801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.131:44907 every 8 connection(s)
I20260811 19:12:25.070719 19600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
I20260811 19:12:25.076699 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 19600
Aug 11 19:12:25 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.131@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:12:25 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.131@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 19:12:25.097604 19394 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:12:25.074160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:44039 (local address 127.3.61.190:35751)
0811 19:12:25.074400 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:25.074405 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:25.076302 (+ 1897us) server_negotiation.cc:407] Connection header received
0811 19:12:25.079425 (+ 3123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:25.079429 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:25.079533 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:25.079690 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:25.083183 (+ 3493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:25.084251 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:25.086756 (+ 2505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:25.087208 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:25.087333 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:25.097228 (+ 9895us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.61.131:44039: BlockingRecv error: recv got EOF from 127.3.61.131:44039 (error 108)
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":90,"spinlock_wait_cycles":8704}
I20260811 19:12:25.098366 19811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:25.074611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35751 (local address 127.3.61.131:44039)
0811 19:12:25.075951 (+ 1340us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:25.076075 (+ 124us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:25.078989 (+ 2914us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:25.080273 (+ 1284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:25.080292 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:25.081343 (+ 1051us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:25.082892 (+ 1549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:25.082945 (+ 53us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:25.084510 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:25.084518 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:25.086563 (+ 2045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:25.086586 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:25.087555 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:25.087566 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:25.087878 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:25.092030 (+ 4152us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:25.092067 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:25.096935 (+ 4868us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.61.190:35751: Server oryx/127.3.61.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":847,"thread_start_us":496,"threads_started":1}
W20260811 19:12:25.099227 19806 heartbeater.cc:644] Failed to heartbeat to 127.3.61.190:35751 (0 consecutive failures): Not authorized: Failed to ping master at 127.3.61.190:35751: Client connection negotiation failed: client connection to 127.3.61.190:35751: Server oryx/127.3.61.190@KRBTEST.COM not found in Kerberos database
I20260811 19:12:25.104561 19813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:25.078225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58240 (local address 127.3.61.190:35751)
0811 19:12:25.079300 (+ 1075us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:25.079305 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:25.079332 (+ 27us) server_negotiation.cc:407] Connection header received
0811 19:12:25.079419 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:25.079425 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:25.079533 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:25.079690 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:25.082409 (+ 2719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:25.083673 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:25.085795 (+ 2122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:25.086334 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:25.086463 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:25.095094 (+ 8631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:25.095106 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:25.095131 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:25.095195 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:25.098762 (+ 3567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:25.100183 (+ 1421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:25.100200 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:25.100216 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:25.100747 (+ 531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:25.101355 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:25.101362 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:25.101365 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:25.102380 (+ 1015us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:25.102702 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:25.103433 (+ 731us) server_negotiation.cc:299] Negotiation successful
0811 19:12:25.103999 (+ 566us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":6,"server-negotiator.queue_time_us":959,"spinlock_wait_cycles":46080,"thread_start_us":784,"threads_started":1}
Aug 11 19:12:25 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.131@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:12:25 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.131@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:12:25.112418 19811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:25.100222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35751 (local address 127.3.61.131:39901)
0811 19:12:25.100481 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:25.100500 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:25.100636 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:25.101089 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:25.101095 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:25.101781 (+ 686us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:25.102050 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:25.102065 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:25.103658 (+ 1593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:25.103665 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:25.105093 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:25.105112 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:25.106265 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:25.106271 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:25.106495 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:25.107356 (+ 861us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:25.107384 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:25.112089 (+ 4705us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.61.190:35751: Server oryx/127.3.61.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":97}
W20260811 19:12:25.112653 19394 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:12:25.100095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:39901 (local address 127.3.61.190:35751)
0811 19:12:25.100382 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:25.100387 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:25.100572 (+ 185us) server_negotiation.cc:407] Connection header received
0811 19:12:25.100768 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:25.100773 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:25.100848 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:25.100967 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:25.102296 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:25.103423 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:25.105287 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:25.106025 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:25.106204 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:25.112376 (+ 6172us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.61.131:39901: BlockingRecv error: recv got EOF from 127.3.61.131:39901 (error 108)
Metrics: {"server-negotiator.queue_time_us":141}
Aug 11 19:12:25 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.131@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:12:25 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.131@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:12:25.124368 19811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:25.113346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35751 (local address 127.3.61.131:55973)
0811 19:12:25.113714 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:25.113745 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:25.113939 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:25.114391 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:25.114396 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:25.115003 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:25.115301 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:25.115320 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:25.116890 (+ 1570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:25.116896 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:25.118024 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:25.118043 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:25.118858 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:25.118865 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:25.119063 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:25.119844 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:25.119865 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:25.124052 (+ 4187us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.61.190:35751: Server oryx/127.3.61.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112}
W20260811 19:12:25.124775 19806 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 19:12:25.124944 19394 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:12:25.113566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:55973 (local address 127.3.61.190:35751)
0811 19:12:25.113819 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:25.113826 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:25.113852 (+ 26us) server_negotiation.cc:407] Connection header received
0811 19:12:25.114147 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:25.114152 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:25.114231 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:25.114327 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:25.115618 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:25.116648 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:25.118229 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:25.118673 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:25.118810 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:25.124455 (+ 5645us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.61.131:55973: BlockingRecv error: recv got EOF from 127.3.61.131:55973 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 11 19:12:25 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.130@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:12:25 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.130@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:12:25.548983 19815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:25.538084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35751 (local address 127.3.61.130:50555)
0811 19:12:25.538859 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:25.538882 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:25.539091 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:25.539485 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:25.539490 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:25.540136 (+ 646us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:25.540371 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:25.540381 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:25.541584 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:25.541591 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:25.542706 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:25.542720 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:25.543630 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:25.543661 (+ 31us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:25.543959 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:25.544905 (+ 946us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:25.544928 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:25.548699 (+ 3771us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.61.190:35751: Server oryx/127.3.61.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":616,"thread_start_us":354,"threads_started":1}
W20260811 19:12:25.549425 19394 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:12:25.538375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:50555 (local address 127.3.61.190:35751)
0811 19:12:25.538736 (+ 361us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:25.538744 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:25.539023 (+ 279us) server_negotiation.cc:407] Connection header received
0811 19:12:25.539252 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:25.539258 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:25.539334 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:25.539433 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:25.540542 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:25.541417 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:25.542861 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:25.543321 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:25.543437 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:25.549050 (+ 5613us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.61.130:50555: BlockingRecv error: recv got EOF from 127.3.61.130:50555 (error 108)
Metrics: {"server-negotiator.queue_time_us":140}
Aug 11 19:12:26 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.129@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:12:26 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.129@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:12:26.030711 19816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:26.018594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35751 (local address 127.3.61.129:57155)
0811 19:12:26.019384 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:26.019407 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:26.019684 (+ 277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:26.020100 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:26.020105 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:26.020769 (+ 664us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:26.020967 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:26.020978 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:26.022848 (+ 1870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:26.022852 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:26.023920 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:26.023935 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:26.024840 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:26.024847 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:26.025039 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:26.026207 (+ 1168us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:26.026236 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:26.030424 (+ 4188us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.61.190:35751: Server oryx/127.3.61.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":651,"thread_start_us":421,"threads_started":1}
W20260811 19:12:26.031064 19394 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:12:26.018859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:57155 (local address 127.3.61.190:35751)
0811 19:12:26.019210 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:26.019219 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:26.019539 (+ 320us) server_negotiation.cc:407] Connection header received
0811 19:12:26.019851 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:26.019855 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:26.019935 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:26.020044 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:26.021485 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:26.022621 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:26.024106 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:26.024649 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:26.024776 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:26.030803 (+ 6027us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.61.129:57155: BlockingRecv error: recv got EOF from 127.3.61.129:57155 (error 108)
Metrics: {"server-negotiator.queue_time_us":117}
Aug 11 19:12:26 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.131@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:12:26 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.131@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:12:26.137202 19817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:26.125841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35751 (local address 127.3.61.131:56671)
0811 19:12:26.126500 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:26.126523 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:26.126693 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:26.127064 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:26.127069 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:26.127697 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:26.127963 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:26.127975 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:26.129129 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:26.129133 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:26.130710 (+ 1577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:26.130747 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:26.131778 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:26.131787 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:26.131967 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:26.132680 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:26.132703 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:26.136871 (+ 4168us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.61.190:35751: Server oryx/127.3.61.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":365,"threads_started":1}
W20260811 19:12:26.137934 19394 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:12:26.125917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:56671 (local address 127.3.61.190:35751)
0811 19:12:26.126176 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:26.126184 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:26.126626 (+ 442us) server_negotiation.cc:407] Connection header received
0811 19:12:26.126838 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:26.126842 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:26.126915 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:26.127002 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:26.128126 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:26.128961 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:26.130991 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:26.131522 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:26.131656 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:26.137438 (+ 5782us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.61.131:56671: BlockingRecv error: recv got EOF from 127.3.61.131:56671 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
Aug 11 19:12:26 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.130@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:12:26 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.130@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:12:26.559787 19818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:26.550211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35751 (local address 127.3.61.130:33427)
0811 19:12:26.550822 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:26.550842 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:26.551005 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:26.551456 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:26.551467 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:26.551947 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:26.552115 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:26.552125 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:26.553261 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:26.553265 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:26.554265 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:26.554281 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:26.555106 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:26.555112 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:26.555274 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:26.555955 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:26.555974 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:26.559523 (+ 3549us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.61.190:35751: Server oryx/127.3.61.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":339,"threads_started":1}
W20260811 19:12:26.560149 19394 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:12:26.550479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:33427 (local address 127.3.61.190:35751)
0811 19:12:26.550766 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:26.550773 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:26.550939 (+ 166us) server_negotiation.cc:407] Connection header received
0811 19:12:26.551163 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:26.551168 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:26.551238 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:26.551331 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:26.552290 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:26.553092 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:26.554438 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:26.554923 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:26.555051 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:26.559883 (+ 4832us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.61.130:33427: BlockingRecv error: recv got EOF from 127.3.61.130:33427 (error 108)
Metrics: {"server-negotiator.queue_time_us":117}
Aug 11 19:12:27 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.129@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:12:27 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.129@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:12:27.043215 19821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:27.031984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35751 (local address 127.3.61.129:45809)
0811 19:12:27.032502 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:27.032525 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:27.032752 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:27.033148 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:27.033152 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:27.033562 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:27.033829 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:27.033841 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:27.035211 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:27.035216 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:27.036185 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:27.036200 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:27.037416 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:27.037424 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:27.037709 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:27.038613 (+ 904us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:27.038635 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:27.042915 (+ 4280us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.61.190:35751: Server oryx/127.3.61.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":283,"threads_started":1}
W20260811 19:12:27.043637 19394 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:12:27.032168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:45809 (local address 127.3.61.190:35751)
0811 19:12:27.032449 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:27.032456 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:27.032626 (+ 170us) server_negotiation.cc:407] Connection header received
0811 19:12:27.032889 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:27.032893 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:27.032964 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:27.033059 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:27.034030 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:27.034964 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:27.036370 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:27.037099 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:27.037241 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:27.043270 (+ 6029us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.61.129:45809: BlockingRecv error: recv got EOF from 127.3.61.129:45809 (error 108)
Metrics: {"server-negotiator.queue_time_us":129}
Aug 11 19:12:27 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.131@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:12:27 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.131@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:12:27.150022 19823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:27.138739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35751 (local address 127.3.61.131:38251)
0811 19:12:27.139429 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:27.139458 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:27.139622 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:27.140226 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:27.140234 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:27.140861 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:27.141123 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:27.141139 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:27.142514 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:27.142520 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:27.143641 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:27.143657 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:27.144592 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:27.144599 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:27.144745 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:27.145480 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:27.145502 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:27.149718 (+ 4216us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.61.190:35751: Server oryx/127.3.61.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":373,"threads_started":1}
W20260811 19:12:27.150399 19394 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:12:27.138886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:38251 (local address 127.3.61.190:35751)
0811 19:12:27.139156 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:27.139165 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:27.139554 (+ 389us) server_negotiation.cc:407] Connection header received
0811 19:12:27.139780 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:27.139786 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:27.139890 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:27.139983 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:27.141361 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:27.142314 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:27.143862 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:27.144384 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:27.144509 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:27.150003 (+ 5494us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.61.131:38251: BlockingRecv error: recv got EOF from 127.3.61.131:38251 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Aug 11 19:12:27 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.130@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:12:27 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.130@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:12:27.571201 19824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:27.561497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35751 (local address 127.3.61.130:40861)
0811 19:12:27.562079 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:27.562099 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:27.562270 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:27.562643 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:27.562648 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:27.563053 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:27.563257 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:27.563272 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:27.564322 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:27.564326 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:27.565281 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:27.565294 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:27.566371 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:27.566376 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:27.566538 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:27.567207 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:27.567229 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:27.570905 (+ 3676us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.61.190:35751: Server oryx/127.3.61.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":313,"threads_started":1}
W20260811 19:12:27.571568 19394 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:12:27.561636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:40861 (local address 127.3.61.190:35751)
0811 19:12:27.561882 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:27.561889 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:27.562200 (+ 311us) server_negotiation.cc:407] Connection header received
0811 19:12:27.562409 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:27.562415 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:27.562494 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:27.562585 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:27.563418 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:27.564171 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:27.565554 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:27.566153 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:27.566311 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:27.571202 (+ 4891us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.61.130:40861: BlockingRecv error: recv got EOF from 127.3.61.130:40861 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 11 19:12:28 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.129@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:12:28 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.129@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:12:28.055225 19825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:28.044833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35751 (local address 127.3.61.129:56813)
0811 19:12:28.045625 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:28.045676 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:28.045881 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:28.046294 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:28.046299 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:28.046882 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:28.047085 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:28.047098 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:28.048602 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:28.048608 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:28.049935 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:28.049962 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:28.050727 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:28.050733 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:28.050929 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:28.051605 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:28.051626 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:28.054970 (+ 3344us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.61.190:35751: Server oryx/127.3.61.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":621,"thread_start_us":444,"threads_started":1}
W20260811 19:12:28.055580 19394 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:12:28.045022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:56813 (local address 127.3.61.190:35751)
0811 19:12:28.045289 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:28.045298 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:28.045788 (+ 490us) server_negotiation.cc:407] Connection header received
0811 19:12:28.046035 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:28.046041 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:28.046118 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:28.046214 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:28.047398 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:28.048302 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:28.050099 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:28.050560 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:28.050684 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:28.055297 (+ 4613us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.61.129:56813: BlockingRecv error: recv got EOF from 127.3.61.129:56813 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 11 19:12:28 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.131@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:12:28 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.131@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:12:28.161919 19827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:28.151458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35751 (local address 127.3.61.131:36193)
0811 19:12:28.152015 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:28.152033 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:28.152182 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:28.152595 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:28.152598 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:28.153003 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:28.153187 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:28.153196 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:28.154572 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:28.154577 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:28.155779 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:28.155796 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:28.156572 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:28.156579 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:28.156744 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:28.157432 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:28.157453 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:28.161517 (+ 4064us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.61.190:35751: Server oryx/127.3.61.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":309,"threads_started":1}
W20260811 19:12:28.162281 19394 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:12:28.151663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:36193 (local address 127.3.61.190:35751)
0811 19:12:28.151961 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:28.151968 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:28.152120 (+ 152us) server_negotiation.cc:407] Connection header received
0811 19:12:28.152351 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:28.152357 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:28.152452 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:28.152548 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:28.153349 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:28.154340 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:28.155958 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:28.156404 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:28.156519 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:28.161894 (+ 5375us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.61.131:36193: BlockingRecv error: recv got EOF from 127.3.61.131:36193 (error 108)
Metrics: {"server-negotiator.queue_time_us":138}
Aug 11 19:12:28 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.130@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:12:28 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.130@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:12:28.582814 19828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:28.572486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35751 (local address 127.3.61.130:55393)
0811 19:12:28.573078 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:28.573094 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:28.573255 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:28.573764 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:28.573769 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:28.574196 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:28.574361 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:28.574370 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:28.575433 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:28.575438 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:28.576443 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:28.576457 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:28.577181 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:28.577187 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:28.577379 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:28.578048 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:28.578067 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:28.582430 (+ 4363us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.61.190:35751: Server oryx/127.3.61.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":334,"threads_started":1}
W20260811 19:12:28.582927 19394 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:12:28.572652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:55393 (local address 127.3.61.190:35751)
0811 19:12:28.572932 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:28.572939 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:28.573184 (+ 245us) server_negotiation.cc:407] Connection header received
0811 19:12:28.573393 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:28.573465 (+ 72us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:28.573553 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:28.573702 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:28.574529 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:28.575294 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:28.576607 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:28.577016 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:28.577141 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:28.582676 (+ 5535us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.61.130:55393: BlockingRecv error: recv got EOF from 127.3.61.130:55393 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 11 19:12:29 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.129@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:12:29 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.129@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:12:29.067518 19829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:29.056691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35751 (local address 127.3.61.129:40803)
0811 19:12:29.057294 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:29.057321 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:29.057474 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:29.057836 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:29.057841 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:29.058375 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:29.058561 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:29.058571 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:29.059579 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:29.059584 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:29.060533 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:29.060547 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:29.061790 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:29.061798 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:29.061966 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:29.062758 (+ 792us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:29.062785 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:29.067175 (+ 4390us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.61.190:35751: Server oryx/127.3.61.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":290,"threads_started":1}
W20260811 19:12:29.067776 19394 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:12:29.056836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:40803 (local address 127.3.61.190:35751)
0811 19:12:29.057068 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:29.057074 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:29.057429 (+ 355us) server_negotiation.cc:407] Connection header received
0811 19:12:29.057597 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:29.057603 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:29.057702 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:29.057789 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:29.058709 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:29.059440 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:29.060818 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:29.061483 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:29.061701 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:29.067481 (+ 5780us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.61.129:40803: BlockingRecv error: recv got EOF from 127.3.61.129:40803 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 11 19:12:29 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.131@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:12:29 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.131@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:12:29.173255 19830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:29.163330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35751 (local address 127.3.61.131:51665)
0811 19:12:29.163905 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:29.163922 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:29.164099 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:29.164448 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:29.164453 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:29.164866 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:29.165026 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:29.165035 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:29.166219 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:29.166224 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:29.167546 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:29.167562 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:29.168366 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:29.168373 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:29.168648 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:29.169353 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:29.169374 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:29.172854 (+ 3480us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.61.190:35751: Server oryx/127.3.61.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":276,"threads_started":1}
W20260811 19:12:29.173789 19394 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:12:29.163452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:51665 (local address 127.3.61.190:35751)
0811 19:12:29.163648 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:29.163656 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:29.164008 (+ 352us) server_negotiation.cc:407] Connection header received
0811 19:12:29.164220 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:29.164225 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:29.164313 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:29.164396 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:29.165169 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:29.165991 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:29.167709 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:29.168184 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:29.168341 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:29.173187 (+ 4846us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.61.131:51665: BlockingRecv error: recv got EOF from 127.3.61.131:51665 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 11 19:12:29 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.130@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:12:29 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.130@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:12:29.593969 19835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:29.584464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35751 (local address 127.3.61.130:37583)
0811 19:12:29.585241 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:29.585264 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:29.585442 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:29.585841 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:29.585846 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:29.586305 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:29.586482 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:29.586492 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:29.587695 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:29.587700 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:29.588638 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:29.588651 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:29.589482 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:29.589488 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:29.589682 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:29.590337 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:29.590357 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:29.593580 (+ 3223us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.61.190:35751: Server oryx/127.3.61.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":629,"thread_start_us":339,"threads_started":1}
W20260811 19:12:29.594123 19394 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:12:29.584845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:37583 (local address 127.3.61.190:35751)
0811 19:12:29.585130 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:29.585137 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:29.585381 (+ 244us) server_negotiation.cc:407] Connection header received
0811 19:12:29.585592 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:29.585596 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:29.585692 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:29.585779 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:29.586638 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:29.587543 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:29.588798 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:29.589314 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:29.589454 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:29.593896 (+ 4442us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.61.130:37583: BlockingRecv error: recv got EOF from 127.3.61.130:37583 (error 108)
Metrics: {"server-negotiator.queue_time_us":113}
Aug 11 19:12:30 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.129@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:12:30 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.129@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:12:30.079689 19836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:30.069473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35751 (local address 127.3.61.129:45443)
0811 19:12:30.070589 (+ 1116us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:30.070619 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:30.070847 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:30.071286 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:30.071292 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:30.071885 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:30.072061 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:30.072071 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:30.073490 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:30.073494 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:30.074513 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:30.074526 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:30.075374 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:30.075379 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:30.075545 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:30.076224 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:30.076243 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:30.079335 (+ 3092us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.61.190:35751: Server oryx/127.3.61.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":923,"thread_start_us":680,"threads_started":1}
W20260811 19:12:30.079941 19394 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:12:30.069952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:45443 (local address 127.3.61.190:35751)
0811 19:12:30.070170 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:30.070176 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:30.070836 (+ 660us) server_negotiation.cc:407] Connection header received
0811 19:12:30.071009 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:30.071015 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:30.071111 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:30.071238 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:30.072250 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:30.073267 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:30.074701 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:30.075167 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:30.075281 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:30.079658 (+ 4377us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.61.129:45443: BlockingRecv error: recv got EOF from 127.3.61.129:45443 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 11 19:12:30 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.131@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:12:30 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.131@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:12:30.188546 19837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:30.174621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35751 (local address 127.3.61.131:35883)
0811 19:12:30.175183 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:30.175199 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:30.175325 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:30.175952 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:30.175956 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:30.176584 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:30.176820 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:30.176834 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:30.178041 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:30.178046 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:30.179012 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:30.179025 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:30.180132 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:30.180139 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:30.180321 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:30.184495 (+ 4174us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:30.184553 (+ 58us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:30.188135 (+ 3582us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.61.190:35751: Server oryx/127.3.61.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":286,"threads_started":1}
W20260811 19:12:30.188961 19394 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:12:30.174656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:35883 (local address 127.3.61.190:35751)
0811 19:12:30.174869 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:30.174875 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:30.175381 (+ 506us) server_negotiation.cc:407] Connection header received
0811 19:12:30.175542 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:30.175549 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:30.175702 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:30.175842 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:30.177019 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:30.177861 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:30.179265 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:30.179823 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:30.179983 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:30.188468 (+ 8485us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.61.131:35883: BlockingRecv error: recv got EOF from 127.3.61.131:35883 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 11 19:12:30 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.130@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:12:30 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.130@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:12:30.606432 19840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:30.595435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35751 (local address 127.3.61.130:58253)
0811 19:12:30.596320 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:30.596341 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:30.596515 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:30.597012 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:30.597018 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:30.597594 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:30.597936 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:30.597952 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:30.599113 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:30.599119 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:30.600042 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:30.600055 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:30.601173 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:30.601182 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:30.601438 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:30.602260 (+ 822us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:30.602282 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:30.606065 (+ 3783us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.61.190:35751: Server oryx/127.3.61.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":713,"thread_start_us":419,"threads_started":1}
W20260811 19:12:30.606658 19394 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:12:30.595688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:58253 (local address 127.3.61.190:35751)
0811 19:12:30.595904 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:30.595909 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:30.596461 (+ 552us) server_negotiation.cc:407] Connection header received
0811 19:12:30.596722 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:30.596727 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:30.596805 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:30.596920 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:30.598133 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:30.598957 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:30.600210 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:30.600919 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:30.601086 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:30.606395 (+ 5309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.61.130:58253: BlockingRecv error: recv got EOF from 127.3.61.130:58253 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 11 19:12:31 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.129@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:12:31 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.129@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:12:31.092547 19841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:31.081368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35751 (local address 127.3.61.129:35067)
0811 19:12:31.082280 (+ 912us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:31.082301 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:31.082471 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:31.083082 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:31.083089 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:31.083650 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:31.083847 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:31.083858 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:31.084966 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:31.084970 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:31.086563 (+ 1593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:31.086583 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:31.087382 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:31.087388 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:31.087562 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:31.088243 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:31.088266 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:31.092143 (+ 3877us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.61.190:35751: Server oryx/127.3.61.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":763,"thread_start_us":468,"threads_started":1}
W20260811 19:12:31.092792 19394 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:12:31.081564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:35067 (local address 127.3.61.190:35751)
0811 19:12:31.081840 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:31.081846 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:31.082533 (+ 687us) server_negotiation.cc:407] Connection header received
0811 19:12:31.082641 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:31.082646 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:31.082738 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:31.082927 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:31.084028 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:31.084800 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:31.086752 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:31.087186 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:31.087330 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:31.092543 (+ 5213us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.61.129:35067: BlockingRecv error: recv got EOF from 127.3.61.129:35067 (error 108)
Metrics: {"server-negotiator.queue_time_us":113}
Aug 11 19:12:31 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.131@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:12:31 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.131@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:12:31.200229 19842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:31.190111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35751 (local address 127.3.61.131:45443)
0811 19:12:31.190653 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:31.190671 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:31.190800 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:31.191212 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:31.191218 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:31.191742 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:31.191919 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:31.191949 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:31.193055 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:31.193059 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:31.194031 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:31.194046 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:31.194867 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:31.194874 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:31.195040 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:31.195786 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:31.195808 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:31.199864 (+ 4056us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.61.190:35751: Server oryx/127.3.61.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":312,"threads_started":1}
W20260811 19:12:31.200505 19394 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:12:31.190228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:45443 (local address 127.3.61.190:35751)
0811 19:12:31.190481 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:31.190488 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:31.190764 (+ 276us) server_negotiation.cc:407] Connection header received
0811 19:12:31.190917 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:31.190923 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:31.191028 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:31.191151 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:31.192090 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:31.192894 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:31.194206 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:31.194668 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:31.194788 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:31.200193 (+ 5405us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.61.131:45443: BlockingRecv error: recv got EOF from 127.3.61.131:45443 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 11 19:12:31 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.130@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:12:31 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.130@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:12:31.618484 19845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:31.607884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35751 (local address 127.3.61.130:54257)
0811 19:12:31.608842 (+ 958us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:31.608867 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:31.609039 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:31.609523 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:31.609527 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:31.610135 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:31.610380 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:31.610390 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:31.611706 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:31.611712 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:31.612732 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:31.612748 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:31.613720 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:31.613735 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:31.613885 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:31.614554 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:31.614574 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:31.618161 (+ 3587us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.61.190:35751: Server oryx/127.3.61.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":800,"thread_start_us":454,"threads_started":1}
W20260811 19:12:31.618654 19394 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:12:31.608042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:54257 (local address 127.3.61.190:35751)
0811 19:12:31.608427 (+ 385us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:31.608436 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:31.609046 (+ 610us) server_negotiation.cc:407] Connection header received
0811 19:12:31.609183 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:31.609188 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:31.609267 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:31.609474 (+ 207us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:31.610608 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:31.611410 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:31.612935 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:31.613390 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:31.613510 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:31.618434 (+ 4924us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.61.130:54257: BlockingRecv error: recv got EOF from 127.3.61.130:54257 (error 108)
Metrics: {"server-negotiator.queue_time_us":190}
Aug 11 19:12:32 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.129@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:12:32 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.129@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:12:32.104599 19846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:32.093938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35751 (local address 127.3.61.129:59159)
0811 19:12:32.094538 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:32.094555 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:32.094710 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:32.095067 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:32.095072 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:32.095566 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:32.095735 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:32.095744 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:32.096863 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:32.096867 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:32.098320 (+ 1453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:32.098350 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:32.099427 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:32.099433 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:32.099598 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:32.100439 (+ 841us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:32.100460 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:32.104321 (+ 3861us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.61.190:35751: Server oryx/127.3.61.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":306,"threads_started":1}
W20260811 19:12:32.104915 19394 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:12:32.094075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:59159 (local address 127.3.61.190:35751)
0811 19:12:32.094334 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:32.094341 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:32.094657 (+ 316us) server_negotiation.cc:407] Connection header received
0811 19:12:32.094837 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:32.094843 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:32.094911 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:32.094998 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:32.095896 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:32.096702 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:32.098556 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:32.099155 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:32.099298 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:32.104616 (+ 5318us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.61.129:59159: BlockingRecv error: recv got EOF from 127.3.61.129:59159 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 11 19:12:32 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.131@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:12:32 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.131@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:12:32.212970 19847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:32.201524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35751 (local address 127.3.61.131:33631)
0811 19:12:32.202391 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:32.202415 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:32.202618 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:32.203041 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:32.203047 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:32.203665 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:32.203884 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:32.203894 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:32.205387 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:32.205395 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:32.206790 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:32.206813 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:32.208471 (+ 1658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:32.208477 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:32.208633 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:32.209425 (+ 792us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:32.209446 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:32.212688 (+ 3242us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.61.190:35751: Server oryx/127.3.61.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":719,"thread_start_us":411,"threads_started":1}
W20260811 19:12:32.213353 19394 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:12:32.202011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:33631 (local address 127.3.61.190:35751)
0811 19:12:32.202310 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:32.202317 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:32.202528 (+ 211us) server_negotiation.cc:407] Connection header received
0811 19:12:32.202769 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:32.202776 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:32.202878 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:32.202981 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:32.204057 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:32.205060 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:32.207115 (+ 2055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:32.208219 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:32.208384 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:32.213112 (+ 4728us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.61.131:33631: BlockingRecv error: recv got EOF from 127.3.61.131:33631 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
Aug 11 19:12:32 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.130@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:12:32 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.130@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:12:32.632117 19848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:32.619869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35751 (local address 127.3.61.130:45889)
0811 19:12:32.620559 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:32.620579 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:32.620774 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:32.621232 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:32.621237 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:32.621930 (+ 693us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:32.622215 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:32.622227 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:32.623713 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:32.623719 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:32.624914 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:32.624934 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:32.626132 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:32.626139 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:32.626298 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:32.627347 (+ 1049us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:32.627371 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:32.631771 (+ 4400us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.61.190:35751: Server oryx/127.3.61.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":352,"threads_started":1}
W20260811 19:12:32.632494 19394 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:12:32.620087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:45889 (local address 127.3.61.190:35751)
0811 19:12:32.620314 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:32.620319 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:32.620694 (+ 375us) server_negotiation.cc:407] Connection header received
0811 19:12:32.620912 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:32.620917 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:32.621007 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:32.621132 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:32.622458 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:32.623514 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:32.625231 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:32.625801 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:32.626002 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:32.632177 (+ 6175us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.61.130:45889: BlockingRecv error: recv got EOF from 127.3.61.130:45889 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Aug 11 19:12:33 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.129@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:12:33 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.129@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:12:33.117097 19850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:33.105989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35751 (local address 127.3.61.129:56545)
0811 19:12:33.106676 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:33.106696 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:33.106855 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:33.107321 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:33.107325 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:33.107911 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:33.108155 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:33.108168 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:33.109719 (+ 1551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:33.109725 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:33.111095 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:33.111118 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:33.112086 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:33.112093 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:33.112272 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:33.113047 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:33.113068 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:33.116721 (+ 3653us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.61.190:35751: Server oryx/127.3.61.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":346,"threads_started":1}
W20260811 19:12:33.117910 19394 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:12:33.106074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:56545 (local address 127.3.61.190:35751)
0811 19:12:33.106305 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:33.106310 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:33.106807 (+ 497us) server_negotiation.cc:407] Connection header received
0811 19:12:33.107027 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:33.107032 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:33.107127 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:33.107229 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:33.108505 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:33.109367 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:33.111357 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:33.111851 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:33.111978 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:33.117176 (+ 5198us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.61.129:56545: BlockingRecv error: recv got EOF from 127.3.61.129:56545 (error 108)
Metrics: {"server-negotiator.queue_time_us":106}
Aug 11 19:12:33 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.131@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:12:33 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.131@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:12:33.225574 19851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:33.214452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35751 (local address 127.3.61.131:45529)
0811 19:12:33.215160 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:33.215180 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:33.215350 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:33.215770 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:33.215774 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:33.216227 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:33.216415 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:33.216424 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:33.217812 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:33.217817 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:33.219005 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:33.219025 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:33.220361 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:33.220370 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:33.220562 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:33.221503 (+ 941us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:33.221541 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:33.225248 (+ 3707us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.61.190:35751: Server oryx/127.3.61.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":420,"threads_started":1}
W20260811 19:12:33.225817 19394 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:12:33.214582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:45529 (local address 127.3.61.190:35751)
0811 19:12:33.214879 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:33.214887 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:33.215323 (+ 436us) server_negotiation.cc:407] Connection header received
0811 19:12:33.215495 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:33.215500 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:33.215571 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:33.215669 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:33.216609 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:33.217569 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:33.219327 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:33.219990 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:33.220174 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:33.225499 (+ 5325us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.61.131:45529: BlockingRecv error: recv got EOF from 127.3.61.131:45529 (error 108)
Metrics: {"server-negotiator.queue_time_us":119}
Aug 11 19:12:33 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.130@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:12:33 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.130@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:12:33.644098 19854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:33.633410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35751 (local address 127.3.61.130:54863)
0811 19:12:33.634051 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:33.634070 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:33.634209 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:33.634609 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:33.634614 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:33.635077 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:33.635261 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:33.635271 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:33.636558 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:33.636563 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:33.637678 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:33.637700 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:33.638677 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:33.638682 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:33.638830 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:33.639572 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:33.639599 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:33.643742 (+ 4143us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.61.190:35751: Server oryx/127.3.61.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":348,"threads_started":1}
W20260811 19:12:33.644403 19394 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:12:33.633783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:54863 (local address 127.3.61.190:35751)
0811 19:12:33.634000 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:33.634007 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:33.634241 (+ 234us) server_negotiation.cc:407] Connection header received
0811 19:12:33.634326 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:33.634330 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:33.634409 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:33.634507 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:33.635492 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:33.636325 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:33.637910 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:33.638437 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:33.638551 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:33.644119 (+ 5568us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.61.130:54863: BlockingRecv error: recv got EOF from 127.3.61.130:54863 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 11 19:12:34 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.129@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:12:34 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.129@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:12:34.128134 19855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:34.118333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35751 (local address 127.3.61.129:41325)
0811 19:12:34.118956 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:34.118972 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:34.119130 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:34.119480 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:34.119485 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:34.119919 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:34.120098 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:34.120108 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:34.121241 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:34.121246 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:34.122275 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:34.122291 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:34.123060 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:34.123066 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:34.123241 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:34.124034 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:34.124054 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:34.127779 (+ 3725us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.61.190:35751: Server oryx/127.3.61.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":348,"threads_started":1}
W20260811 19:12:34.128471 19394 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:12:34.118560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:41325 (local address 127.3.61.190:35751)
0811 19:12:34.118844 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:34.118850 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:34.119075 (+ 225us) server_negotiation.cc:407] Connection header received
0811 19:12:34.119256 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:34.119262 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:34.119329 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:34.119433 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:34.120277 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:34.121072 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:34.122455 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:34.122881 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:34.123002 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:34.128159 (+ 5157us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.61.129:41325: BlockingRecv error: recv got EOF from 127.3.61.129:41325 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
Aug 11 19:12:34 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.131@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:12:34 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.131@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:12:34.238685 19856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:34.227083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35751 (local address 127.3.61.131:36571)
0811 19:12:34.227765 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:34.227788 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:34.227979 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:34.228546 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:34.228551 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:34.229333 (+ 782us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:34.229940 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:34.229962 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:34.231749 (+ 1787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:34.231757 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:34.232823 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:34.232841 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:34.233902 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:34.233913 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:34.234073 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:34.234774 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:34.234794 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:34.238397 (+ 3603us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.61.190:35751: Server oryx/127.3.61.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":467,"threads_started":1}
W20260811 19:12:34.238981 19394 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:12:34.227302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:36571 (local address 127.3.61.190:35751)
0811 19:12:34.227557 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:34.227565 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:34.227899 (+ 334us) server_negotiation.cc:407] Connection header received
0811 19:12:34.228111 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:34.228117 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:34.228209 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:34.228301 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:34.230303 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:34.231452 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:34.233053 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:34.233528 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:34.233689 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:34.238736 (+ 5047us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.61.131:36571: BlockingRecv error: recv got EOF from 127.3.61.131:36571 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 11 19:12:34 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.130@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:12:34 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.130@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:12:34.655529 19858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:34.645267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35751 (local address 127.3.61.130:36681)
0811 19:12:34.646075 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:34.646105 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:34.646292 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:34.646729 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:34.646735 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:34.647242 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:34.647461 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:34.647472 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:34.648543 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:34.648547 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:34.649509 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:34.649523 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:34.650351 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:34.650357 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:34.650540 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:34.651338 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:34.651359 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:34.655140 (+ 3781us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.61.190:35751: Server oryx/127.3.61.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":642,"thread_start_us":595,"threads_started":1}
W20260811 19:12:34.655769 19394 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:12:34.645567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:36681 (local address 127.3.61.190:35751)
0811 19:12:34.645847 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:34.645853 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:34.646243 (+ 390us) server_negotiation.cc:407] Connection header received
0811 19:12:34.646447 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:34.646453 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:34.646541 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:34.646648 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:34.647621 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:34.648391 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:34.649707 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:34.650172 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:34.650294 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:34.655506 (+ 5212us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.61.130:36681: BlockingRecv error: recv got EOF from 127.3.61.130:36681 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 11 19:12:35 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.129@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:12:35 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.129@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:12:35.139861 19859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:35.129479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35751 (local address 127.3.61.129:40781)
0811 19:12:35.130306 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:35.130328 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:35.130507 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:35.130967 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:35.130971 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:35.131515 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:35.131940 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:35.131968 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:35.133131 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:35.133136 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:35.134181 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:35.134197 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:35.135056 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:35.135062 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:35.135197 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:35.135918 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:35.135938 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:35.139495 (+ 3557us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.61.190:35751: Server oryx/127.3.61.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":677,"thread_start_us":501,"threads_started":1}
W20260811 19:12:35.139992 19394 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:12:35.129898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:40781 (local address 127.3.61.190:35751)
0811 19:12:35.130294 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:35.130300 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:35.130482 (+ 182us) server_negotiation.cc:407] Connection header received
0811 19:12:35.130654 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:35.130659 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:35.130739 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:35.130858 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:35.132189 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:35.132951 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:35.134375 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:35.134838 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:35.134954 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:35.139731 (+ 4777us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.61.129:40781: BlockingRecv error: recv got EOF from 127.3.61.129:40781 (error 108)
Metrics: {"server-negotiator.queue_time_us":150}
Aug 11 19:12:35 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.131@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:12:35 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.131@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:12:35.255291 19861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:35.240384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35751 (local address 127.3.61.131:60193)
0811 19:12:35.241014 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:35.241033 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:35.241185 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:35.241677 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:35.241684 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:35.242295 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:35.242546 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:35.242561 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:35.244356 (+ 1795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:35.244362 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:35.245452 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:35.245470 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:35.246613 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:35.246621 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:35.246798 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:35.251739 (+ 4941us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:35.251767 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:35.254996 (+ 3229us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.61.190:35751: Server oryx/127.3.61.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":349,"threads_started":1}
W20260811 19:12:35.256071 19394 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:12:35.240554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:60193 (local address 127.3.61.190:35751)
0811 19:12:35.240777 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:35.240782 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:35.241121 (+ 339us) server_negotiation.cc:407] Connection header received
0811 19:12:35.241320 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:35.241326 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:35.241424 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:35.241558 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:35.242787 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:35.244049 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:35.245729 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:35.246337 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:35.246486 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:35.255740 (+ 9254us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.61.131:60193: BlockingRecv error: recv got EOF from 127.3.61.131:60193 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 11 19:12:35 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.130@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:12:35 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.130@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:12:35.668005 19862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:35.657004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35751 (local address 127.3.61.130:49455)
0811 19:12:35.657559 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:35.657577 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:35.657747 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:35.658125 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:35.658129 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:35.658706 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:35.658900 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:35.658909 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:35.660011 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:35.660015 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:35.661300 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:35.661334 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:35.662688 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:35.662695 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:35.662927 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:35.663896 (+ 969us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:35.663929 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:35.667698 (+ 3769us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.61.190:35751: Server oryx/127.3.61.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":289,"threads_started":1}
W20260811 19:12:35.668308 19394 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:12:35.657146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:49455 (local address 127.3.61.190:35751)
0811 19:12:35.657485 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:35.657492 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:35.657702 (+ 210us) server_negotiation.cc:407] Connection header received
0811 19:12:35.657874 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:35.657879 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:35.657955 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:35.658047 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:35.659069 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:35.659843 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:35.661765 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:35.662419 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:35.662590 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:35.668037 (+ 5447us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.61.130:49455: BlockingRecv error: recv got EOF from 127.3.61.130:49455 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
Aug 11 19:12:36 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.129@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:12:36 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.129@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:12:36.152980 19863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:36.141627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35751 (local address 127.3.61.129:37295)
0811 19:12:36.142257 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:36.142275 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:36.142458 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:36.142920 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:36.142926 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:36.143687 (+ 761us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:36.144003 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:36.144022 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:36.145974 (+ 1952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:36.145979 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:36.147079 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:36.147096 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:36.148089 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:36.148097 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:36.148329 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:36.149138 (+ 809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:36.149160 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:36.152718 (+ 3558us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.61.190:35751: Server oryx/127.3.61.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":313,"threads_started":1}
W20260811 19:12:36.153323 19394 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:12:36.141727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:37295 (local address 127.3.61.190:35751)
0811 19:12:36.141992 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:36.141999 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:36.142502 (+ 503us) server_negotiation.cc:407] Connection header received
0811 19:12:36.142590 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:36.142596 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:36.142678 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:36.142799 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:36.144263 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:36.145734 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:36.147280 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:36.147798 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:36.147914 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:36.153050 (+ 5136us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.61.129:37295: BlockingRecv error: recv got EOF from 127.3.61.129:37295 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 11 19:12:36 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.131@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:12:36 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.131@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:12:36.268914 19866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:36.256782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35751 (local address 127.3.61.131:34679)
0811 19:12:36.257455 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:36.257474 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:36.257881 (+ 407us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:36.258272 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:36.258277 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:36.258825 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:36.259018 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:36.259027 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:36.260741 (+ 1714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:36.260749 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:36.261905 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:36.261923 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:36.262858 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:36.262865 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:36.263022 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:36.263955 (+ 933us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:36.263977 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:36.268464 (+ 4487us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.61.190:35751: Server oryx/127.3.61.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":380,"threads_started":1}
W20260811 19:12:36.269116 19394 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:12:36.256986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:34679 (local address 127.3.61.190:35751)
0811 19:12:36.257300 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:36.257307 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:36.257568 (+ 261us) server_negotiation.cc:407] Connection header received
0811 19:12:36.258038 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:36.258044 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:36.258135 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:36.258241 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:36.259390 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:36.260525 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:36.262088 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:36.262598 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:36.262767 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:36.268814 (+ 6047us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.61.131:34679: BlockingRecv error: recv got EOF from 127.3.61.131:34679 (error 108)
Metrics: {"server-negotiator.queue_time_us":103}
Aug 11 19:12:36 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.130@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:12:36 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.130@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:12:36.687258 19867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:36.669374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35751 (local address 127.3.61.130:36103)
0811 19:12:36.669936 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:36.669956 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:36.670112 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:36.670509 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:36.670512 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:36.670981 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:36.671154 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:36.671163 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:36.672519 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:36.672524 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:36.673476 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:36.673490 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:36.674393 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:36.674398 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:36.674542 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:36.682876 (+ 8334us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:36.682912 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:36.686947 (+ 4035us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.61.190:35751: Server oryx/127.3.61.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":460,"threads_started":1}
W20260811 19:12:36.687628 19394 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:12:36.669523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:36103 (local address 127.3.61.190:35751)
0811 19:12:36.669827 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:36.669834 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:36.670074 (+ 240us) server_negotiation.cc:407] Connection header received
0811 19:12:36.670253 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:36.670258 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:36.670331 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:36.670427 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:36.671308 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:36.672248 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:36.673677 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:36.674221 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:36.674333 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:36.687279 (+ 12946us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.61.130:36103: BlockingRecv error: recv got EOF from 127.3.61.130:36103 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 11 19:12:37 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.129@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:12:37 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.129@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:12:37.165284 19869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:37.154117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35751 (local address 127.3.61.129:60787)
0811 19:12:37.154654 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:37.154672 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:37.154806 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:37.155096 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:37.155099 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:37.155529 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:37.155709 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:37.155717 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:37.157307 (+ 1590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:37.157314 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:37.158450 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:37.158471 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:37.159380 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:37.159386 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:37.159528 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:37.160299 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:37.160328 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:37.164934 (+ 4606us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.61.190:35751: Server oryx/127.3.61.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":283,"threads_started":1}
W20260811 19:12:37.165619 19394 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:12:37.154201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:60787 (local address 127.3.61.190:35751)
0811 19:12:37.154433 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:37.154440 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:37.154803 (+ 363us) server_negotiation.cc:407] Connection header received
0811 19:12:37.154908 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:37.154911 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:37.154978 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:37.155059 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:37.156005 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:37.157095 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:37.158662 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:37.159152 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:37.159265 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:37.165290 (+ 6025us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.61.129:60787: BlockingRecv error: recv got EOF from 127.3.61.129:60787 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 11 19:12:37 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.131@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:12:37 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.131@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:12:37.280974 19870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:37.270437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35751 (local address 127.3.61.131:33273)
0811 19:12:37.270997 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:37.271014 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:37.271151 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:37.271679 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:37.271683 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:37.272124 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:37.272315 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:37.272324 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:37.273626 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:37.273630 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:37.274876 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:37.274896 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:37.275927 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:37.275933 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:37.276129 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:37.276939 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:37.276966 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:37.280683 (+ 3717us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.61.190:35751: Server oryx/127.3.61.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":307,"threads_started":1}
W20260811 19:12:37.281572 19394 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:12:37.270614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:33273 (local address 127.3.61.190:35751)
0811 19:12:37.270909 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:37.270916 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:37.271183 (+ 267us) server_negotiation.cc:407] Connection header received
0811 19:12:37.271309 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:37.271314 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:37.271416 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:37.271568 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:37.272558 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:37.273427 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:37.275128 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:37.275740 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:37.275889 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:37.281217 (+ 5328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.61.131:33273: BlockingRecv error: recv got EOF from 127.3.61.131:33273 (error 108)
Metrics: {"server-negotiator.queue_time_us":144}
Aug 11 19:12:37 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.130@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:12:37 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.130@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:12:37.699496 19871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:37.688539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35751 (local address 127.3.61.130:57113)
0811 19:12:37.689385 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:37.689411 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:37.689572 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:37.689953 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:37.689956 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:37.690395 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:37.690566 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:37.690574 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:37.692195 (+ 1621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:37.692200 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:37.693264 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:37.693281 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:37.694304 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:37.694310 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:37.694467 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:37.695143 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:37.695164 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:37.699192 (+ 4028us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.61.190:35751: Server oryx/127.3.61.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":703,"thread_start_us":414,"threads_started":1}
W20260811 19:12:37.699993 19394 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:12:37.688764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:57113 (local address 127.3.61.190:35751)
0811 19:12:37.688989 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:37.688994 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:37.689599 (+ 605us) server_negotiation.cc:407] Connection header received
0811 19:12:37.689727 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:37.689733 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:37.689816 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:37.689916 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:37.690811 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:37.691985 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:37.693506 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:37.694065 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:37.694185 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:37.699650 (+ 5465us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.61.130:57113: BlockingRecv error: recv got EOF from 127.3.61.130:57113 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 11 19:12:38 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.129@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:12:38 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.129@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:12:38.178210 19872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:38.166944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35751 (local address 127.3.61.129:46215)
0811 19:12:38.167631 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:38.167651 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:38.167811 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:38.168300 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:38.168304 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:38.168884 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:38.169309 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:38.169328 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:38.170908 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:38.170914 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:38.171896 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:38.171911 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:38.172740 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:38.172746 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:38.172932 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:38.173743 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:38.173763 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:38.177834 (+ 4071us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.61.190:35751: Server oryx/127.3.61.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":404,"threads_started":1}
W20260811 19:12:38.178311 19394 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:12:38.167101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:46215 (local address 127.3.61.190:35751)
0811 19:12:38.167397 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:38.167405 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:38.167857 (+ 452us) server_negotiation.cc:407] Connection header received
0811 19:12:38.167968 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:38.167973 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:38.168062 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:38.168193 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:38.169634 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:38.170638 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:38.172079 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:38.172542 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:38.172649 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:38.178063 (+ 5414us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.61.129:46215: BlockingRecv error: recv got EOF from 127.3.61.129:46215 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 11 19:12:38 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.131@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:12:38 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.131@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:12:38.293865 19874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:38.282777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35751 (local address 127.3.61.131:36059)
0811 19:12:38.283652 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:38.283679 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:38.283871 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:38.284470 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:38.284477 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:38.285372 (+ 895us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:38.285610 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:38.285623 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:38.286963 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:38.286969 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:38.288075 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:38.288093 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:38.289300 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:38.289308 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:38.289499 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:38.290280 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:38.290307 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:38.293554 (+ 3247us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.61.190:35751: Server oryx/127.3.61.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":728,"thread_start_us":456,"threads_started":1}
W20260811 19:12:38.294164 19394 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:12:38.282948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:36059 (local address 127.3.61.190:35751)
0811 19:12:38.283478 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:38.283485 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:38.283871 (+ 386us) server_negotiation.cc:407] Connection header received
0811 19:12:38.284120 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:38.284127 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:38.284226 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:38.284317 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:38.285828 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:38.286761 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:38.288323 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:38.288992 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:38.289119 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:38.293906 (+ 4787us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.61.131:36059: BlockingRecv error: recv got EOF from 127.3.61.131:36059 (error 108)
Metrics: {"server-negotiator.queue_time_us":275}
Aug 11 19:12:38 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.130@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:12:38 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.130@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:12:38.711598 19875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:38.701005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35751 (local address 127.3.61.130:34597)
0811 19:12:38.701810 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:38.701832 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:38.702040 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:38.702464 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:38.702468 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:38.702988 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:38.703224 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:38.703235 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:38.704469 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:38.704473 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:38.705483 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:38.705497 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:38.706364 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:38.706370 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:38.706562 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:38.707382 (+ 820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:38.707403 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:38.711338 (+ 3935us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.61.190:35751: Server oryx/127.3.61.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":675,"thread_start_us":385,"threads_started":1}
W20260811 19:12:38.712435 19394 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:12:38.701247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:34597 (local address 127.3.61.190:35751)
0811 19:12:38.701602 (+ 355us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:38.701609 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:38.701952 (+ 343us) server_negotiation.cc:407] Connection header received
0811 19:12:38.702209 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:38.702214 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:38.702302 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:38.702398 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:38.703402 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:38.704296 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:38.705720 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:38.706189 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:38.706317 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:38.712136 (+ 5819us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.61.130:34597: BlockingRecv error: recv got EOF from 127.3.61.130:34597 (error 108)
Metrics: {"server-negotiator.queue_time_us":133}
Aug 11 19:12:39 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.129@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:12:39 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.129@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:12:39.191452 19876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:39.179750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35751 (local address 127.3.61.129:58441)
0811 19:12:39.180338 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:39.180357 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:39.180487 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:39.180986 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:39.180992 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:39.181582 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:39.181918 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:39.181930 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:39.183267 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:39.183272 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:39.184422 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:39.184438 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:39.185459 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:39.185465 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:39.185986 (+ 521us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:39.186701 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:39.186720 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:39.190885 (+ 4165us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.61.190:35751: Server oryx/127.3.61.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":301,"threads_started":1}
W20260811 19:12:39.191617 19394 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:12:39.180086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:58441 (local address 127.3.61.190:35751)
0811 19:12:39.180328 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:39.180333 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:39.180494 (+ 161us) server_negotiation.cc:407] Connection header received
0811 19:12:39.180599 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:39.180604 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:39.180691 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:39.180821 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:39.182126 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:39.183077 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:39.184671 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:39.185206 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:39.185375 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:39.191265 (+ 5890us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.61.129:58441: BlockingRecv error: recv got EOF from 127.3.61.129:58441 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 11 19:12:39 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.131@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:12:39 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.131@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:12:39.305768 19878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:39.295437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35751 (local address 127.3.61.131:53857)
0811 19:12:39.295990 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:39.296014 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:39.296175 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:39.296606 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:39.296610 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:39.297169 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:39.297388 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:39.297404 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:39.298750 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:39.298755 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:39.299745 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:39.299760 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:39.300579 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:39.300585 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:39.300759 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:39.301710 (+ 951us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:39.301735 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:39.305369 (+ 3634us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.61.190:35751: Server oryx/127.3.61.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":292,"threads_started":1}
W20260811 19:12:39.306022 19394 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:12:39.295585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:53857 (local address 127.3.61.190:35751)
0811 19:12:39.295837 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:39.295845 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:39.296131 (+ 286us) server_negotiation.cc:407] Connection header received
0811 19:12:39.296287 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:39.296291 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:39.296374 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:39.296483 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:39.297614 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:39.298562 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:39.299918 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:39.300396 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:39.300545 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:39.305766 (+ 5221us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.61.131:53857: BlockingRecv error: recv got EOF from 127.3.61.131:53857 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Aug 11 19:12:39 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.130@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 19:12:39 dist-test-slave-t7g5 krb5kdc[18187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.61.130@KRBTEST.COM for oryx/127.3.61.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 19:12:39.723915 19880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:39.713094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35751 (local address 127.3.61.130:45239)
0811 19:12:39.713645 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:39.713702 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:39.713863 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:39.714257 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:39.714261 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:39.714728 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:39.715110 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:39.715127 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:39.716264 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:39.716269 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:39.717317 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:39.717333 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:39.718439 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:39.718452 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:39.718641 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:39.719533 (+ 892us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:39.719556 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:39.723566 (+ 4010us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.61.190:35751: Server oryx/127.3.61.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":300,"threads_started":1}
W20260811 19:12:39.724270 19394 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:12:39.713334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:45239 (local address 127.3.61.190:35751)
0811 19:12:39.713554 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:39.713559 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:39.713860 (+ 301us) server_negotiation.cc:407] Connection header received
0811 19:12:39.713985 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:39.713989 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:39.714059 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:39.714156 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:39.715303 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:39.716049 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:39.717544 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:39.718216 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:39.718345 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:39.723970 (+ 5625us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.61.130:45239: BlockingRecv error: recv got EOF from 127.3.61.130:45239 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 19:12:40.083635 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 19145
W20260811 19:12:40.099934 3318 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 19145: Not authorized: Client connection negotiation failed: client connection to 127.3.61.129:42011: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.3.61.129@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 19:12:40.100078 19882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:40.085374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45014 (local address 127.3.61.129:42011)
0811 19:12:40.086177 (+ 803us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:40.086187 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:40.086222 (+ 35us) server_negotiation.cc:407] Connection header received
0811 19:12:40.086305 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:40.086314 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:40.086626 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:40.086822 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:40.088680 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:40.089979 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:40.091906 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:40.092717 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:40.092852 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:40.095852 (+ 3000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:40.095864 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:40.095883 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:40.095933 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:40.098829 (+ 2896us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.3.61.129@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 19:12:40.099686 (+ 857us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45014: Request ticket server kudu/127.3.61.129@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":412,"threads_started":1}
W20260811 19:12:40.100225 19882 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45014: Request ticket server kudu/127.3.61.129@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 19:12:40.100255 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 19145
I20260811 19:12:40.147719 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 19390
I20260811 19:12:40.164518 19883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:40.149563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34794 (local address 127.3.61.130:46301)
0811 19:12:40.150263 (+ 700us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:40.150270 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:40.150303 (+ 33us) server_negotiation.cc:407] Connection header received
0811 19:12:40.150367 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:40.150374 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:40.150625 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:40.150781 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:40.152288 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:40.153405 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:40.155036 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:40.155863 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:40.155990 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:40.159854 (+ 3864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:40.159871 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:40.159895 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:40.160000 (+ 105us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:40.163848 (+ 3848us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.3.61.130@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 19:12:40.164215 (+ 367us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34794: Request ticket server kudu/127.3.61.130@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":530,"thread_start_us":376,"threads_started":1}
W20260811 19:12:40.164628 19883 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34794: Request ticket server kudu/127.3.61.130@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260811 19:12:40.164959 3318 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 19390: Not authorized: Client connection negotiation failed: client connection to 127.3.61.130:46301: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.3.61.130@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 19:12:40.165138 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 19390
I20260811 19:12:40.203572 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 19600
I20260811 19:12:40.220463 19884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:40.205174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36376 (local address 127.3.61.131:44907)
0811 19:12:40.205966 (+ 792us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:40.205975 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:40.206009 (+ 34us) server_negotiation.cc:407] Connection header received
0811 19:12:40.206104 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:40.206112 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:40.206419 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:40.206878 (+ 459us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:40.208966 (+ 2088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:40.210571 (+ 1605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:40.212090 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:40.212798 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:40.212940 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:40.216483 (+ 3543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:40.216500 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:40.216523 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:40.216587 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:40.219676 (+ 3089us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.3.61.131@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 19:12:40.220105 (+ 429us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36376: Request ticket server kudu/127.3.61.131@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":378,"threads_started":1}
W20260811 19:12:40.220594 19884 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36376: Request ticket server kudu/127.3.61.131@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260811 19:12:40.220944 3318 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 19600: Not authorized: Client connection negotiation failed: client connection to 127.3.61.131:44907: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.3.61.131@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 19:12:40.221156 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 19600
I20260811 19:12:40.247372 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 19035
I20260811 19:12:40.249589 19135 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:12:40.385948 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 19035
2026-08-11T19:12:40Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (22424 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-11T19:12:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:12:40Z Disabled control of system clock
I20260811 19:12:40.515288 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:44367
--webserver_interface=127.3.61.190
--webserver_port=0
--builtin_ntp_servers=127.3.61.148:43635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.61.190:44367
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 19:12:40.941043 19894 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:12:40.941417 19894 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:12:40.941515 19894 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:12:40.950873 19894 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:12:40.951035 19894 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:12:40.951073 19894 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:12:40.951138 19894 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:12:40.951201 19894 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:12:40.961412 19894 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:43635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.61.190:44367
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:44367
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.61.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:12:40.963002 19894 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:12:40.964900 19894 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:12:40.978948 19900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:40.979276 19901 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:40.979276 19904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:12:40.981527 19894 server_base.cc:1034] running on GCE node
I20260811 19:12:40.982630 19894 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:12:40.984547 19894 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:12:40.985914 19894 hybrid_clock.cc:648] HybridClock initialized: now 1786475560985825 us; error 49 us; skew 500 ppm
I20260811 19:12:40.986534 19894 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 19:12:40.994575 19894 webserver.cc:460] Webserver started at http://127.3.61.190:34637/ using document root <none> and password file <none>
I20260811 19:12:40.995635 19894 fs_manager.cc:359] Metadata directory not provided
I20260811 19:12:40.995795 19894 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:12:40.996177 19894 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:12:40.999684 19894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/master-0/data/instance:
uuid: "dca6635e3dc54a148026993b68c03e1b"
format_stamp: "Formatted at 2026-08-11 19:12:40 on dist-test-slave-t7g5"
server_key: "66d4b4ea1ab88c1cd2d3e0ed573d7b37"
server_key_iv: "c5178390e5f68f9f91c5d7196ee4e9ed"
server_key_version: "encryptionkey@0"
I20260811 19:12:41.000609 19894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/master-0/wal/instance:
uuid: "dca6635e3dc54a148026993b68c03e1b"
format_stamp: "Formatted at 2026-08-11 19:12:40 on dist-test-slave-t7g5"
server_key: "66d4b4ea1ab88c1cd2d3e0ed573d7b37"
server_key_iv: "c5178390e5f68f9f91c5d7196ee4e9ed"
server_key_version: "encryptionkey@0"
I20260811 19:12:41.007159 19894 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260811 19:12:41.013530 19911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:41.015484 19894 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260811 19:12:41.015702 19894 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/master-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/master-0/wal
uuid: "dca6635e3dc54a148026993b68c03e1b"
format_stamp: "Formatted at 2026-08-11 19:12:40 on dist-test-slave-t7g5"
server_key: "66d4b4ea1ab88c1cd2d3e0ed573d7b37"
server_key_iv: "c5178390e5f68f9f91c5d7196ee4e9ed"
server_key_version: "encryptionkey@0"
I20260811 19:12:41.015946 19894 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:12:41.033918 19894 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:12:41.035166 19894 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:12:41.035501 19894 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:12:41.059128 19894 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.190:44367
I20260811 19:12:41.059130 20003 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.190:44367 every 8 connection(s)
I20260811 19:12:41.060793 19894 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
I20260811 19:12:41.066049 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 19894
I20260811 19:12:41.066144 20004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:12:41.066253 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/master-0/wal/instance
I20260811 19:12:41.066962 3318 external_mini_cluster.cc:1428] Setting key 4cfe9ec03092a636f8f9cac77d17511d
I20260811 19:12:41.082626 20004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dca6635e3dc54a148026993b68c03e1b: Bootstrap starting.
I20260811 19:12:41.086436 20010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:41.069206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35180 (local address 127.3.61.190:44367)
0811 19:12:41.071039 (+ 1833us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:41.071052 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:41.071112 (+ 60us) server_negotiation.cc:407] Connection header received
0811 19:12:41.073467 (+ 2355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:41.073512 (+ 45us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:41.074443 (+ 931us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:41.075303 (+ 860us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:41.076262 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:41.077793 (+ 1531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:41.079443 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:41.080775 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:41.081090 (+ 315us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:41.082929 (+ 1839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:41.082945 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:41.082971 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:12:41.083792 (+ 821us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:41.083919 (+ 127us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:12:41.083950 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:41.084097 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:41.084450 (+ 353us) server_negotiation.cc:299] Negotiation successful
0811 19:12:41.084944 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1105,"thread_start_us":487,"threads_started":1}
I20260811 19:12:41.086845 20004 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dca6635e3dc54a148026993b68c03e1b: Neither blocks nor log segments found. Creating new log.
I20260811 19:12:41.088356 20004 log.cc:824] T 00000000000000000000000000000000 P dca6635e3dc54a148026993b68c03e1b: Log is configured to *not* fsync() on all Append() calls
I20260811 19:12:41.091979 20004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dca6635e3dc54a148026993b68c03e1b: No bootstrap required, opened a new log
I20260811 19:12:41.101261 20004 raft_consensus.cc:348] T 00000000000000000000000000000000 P dca6635e3dc54a148026993b68c03e1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dca6635e3dc54a148026993b68c03e1b" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 44367 } }
I20260811 19:12:41.101857 20004 raft_consensus.cc:374] T 00000000000000000000000000000000 P dca6635e3dc54a148026993b68c03e1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:12:41.101998 20004 raft_consensus.cc:729] T 00000000000000000000000000000000 P dca6635e3dc54a148026993b68c03e1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dca6635e3dc54a148026993b68c03e1b, State: Initialized, Role: FOLLOWER
I20260811 19:12:41.102782 20004 consensus_queue.cc:260] T 00000000000000000000000000000000 P dca6635e3dc54a148026993b68c03e1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dca6635e3dc54a148026993b68c03e1b" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 44367 } }
I20260811 19:12:41.103065 20004 raft_consensus.cc:388] T 00000000000000000000000000000000 P dca6635e3dc54a148026993b68c03e1b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:12:41.103193 20004 raft_consensus.cc:482] T 00000000000000000000000000000000 P dca6635e3dc54a148026993b68c03e1b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:12:41.103401 20004 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dca6635e3dc54a148026993b68c03e1b [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:12:41.106777 20004 raft_consensus.cc:504] T 00000000000000000000000000000000 P dca6635e3dc54a148026993b68c03e1b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dca6635e3dc54a148026993b68c03e1b" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 44367 } }
I20260811 19:12:41.107604 20004 leader_election.cc:302] T 00000000000000000000000000000000 P dca6635e3dc54a148026993b68c03e1b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dca6635e3dc54a148026993b68c03e1b; no voters:
I20260811 19:12:41.108489 20004 leader_election.cc:288] T 00000000000000000000000000000000 P dca6635e3dc54a148026993b68c03e1b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:12:41.108970 20014 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dca6635e3dc54a148026993b68c03e1b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:12:41.110092 20014 raft_consensus.cc:686] T 00000000000000000000000000000000 P dca6635e3dc54a148026993b68c03e1b [term 1 LEADER]: Becoming Leader. State: Replica: dca6635e3dc54a148026993b68c03e1b, State: Running, Role: LEADER
I20260811 19:12:41.110885 20014 consensus_queue.cc:237] T 00000000000000000000000000000000 P dca6635e3dc54a148026993b68c03e1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dca6635e3dc54a148026993b68c03e1b" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 44367 } }
I20260811 19:12:41.111701 20004 sys_catalog.cc:564] T 00000000000000000000000000000000 P dca6635e3dc54a148026993b68c03e1b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:12:41.114218 20016 sys_catalog.cc:455] T 00000000000000000000000000000000 P dca6635e3dc54a148026993b68c03e1b [sys.catalog]: SysCatalogTable state changed. Reason: New leader dca6635e3dc54a148026993b68c03e1b. Latest consensus state: current_term: 1 leader_uuid: "dca6635e3dc54a148026993b68c03e1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dca6635e3dc54a148026993b68c03e1b" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 44367 } } }
I20260811 19:12:41.114727 20016 sys_catalog.cc:458] T 00000000000000000000000000000000 P dca6635e3dc54a148026993b68c03e1b [sys.catalog]: This master's current role is: LEADER
I20260811 19:12:41.115414 20015 sys_catalog.cc:455] T 00000000000000000000000000000000 P dca6635e3dc54a148026993b68c03e1b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dca6635e3dc54a148026993b68c03e1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dca6635e3dc54a148026993b68c03e1b" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 44367 } } }
I20260811 19:12:41.115923 20015 sys_catalog.cc:458] T 00000000000000000000000000000000 P dca6635e3dc54a148026993b68c03e1b [sys.catalog]: This master's current role is: LEADER
I20260811 19:12:41.117303 20024 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:12:41.125097 20024 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:12:41.137719 20024 catalog_manager.cc:1347] Generated new cluster ID: 928069269b7d49ad89eb6be37eb7d75d
I20260811 19:12:41.137871 20024 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:12:41.166611 20024 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:12:41.168058 20024 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:12:41.179538 20024 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dca6635e3dc54a148026993b68c03e1b: Generated new TSK 0
I20260811 19:12:41.180734 20024 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:12:41.191084 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.129:0
--local_ip_for_outbound_sockets=127.3.61.129
--webserver_interface=127.3.61.129
--webserver_port=0
--tserver_master_addrs=127.3.61.190:44367
--builtin_ntp_servers=127.3.61.148:43635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260811 19:12:41.612195 20048 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:12:41.612564 20048 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:12:41.612675 20048 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:12:41.621320 20048 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:12:41.621503 20048 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:12:41.621831 20048 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.129
I20260811 19:12:41.632365 20048 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:43635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.61.129
--webserver_port=0
--tserver_master_addrs=127.3.61.190:44367
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.129
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:12:41.633906 20048 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:12:41.637247 20048 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:12:41.654665 20056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:41.654713 20059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:41.654626 20057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:12:41.656592 20048 server_base.cc:1034] running on GCE node
I20260811 19:12:41.657431 20048 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:12:41.658779 20048 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:12:41.660230 20048 hybrid_clock.cc:648] HybridClock initialized: now 1786475561660070 us; error 159 us; skew 500 ppm
I20260811 19:12:41.660858 20048 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 19:12:41.664672 20048 webserver.cc:460] Webserver started at http://127.3.61.129:35595/ using document root <none> and password file <none>
I20260811 19:12:41.665851 20048 fs_manager.cc:359] Metadata directory not provided
I20260811 19:12:41.666183 20048 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:12:41.666796 20048 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:12:41.670365 20048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-0/data/instance:
uuid: "025171d21e634217b58b1f0fa00e0a21"
format_stamp: "Formatted at 2026-08-11 19:12:41 on dist-test-slave-t7g5"
server_key: "8a9607b443d9ae0c019c90fde11da9ff"
server_key_iv: "72b62dc3a14835a70981dc38c3dda052"
server_key_version: "encryptionkey@0"
I20260811 19:12:41.671365 20048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-0/wal/instance:
uuid: "025171d21e634217b58b1f0fa00e0a21"
format_stamp: "Formatted at 2026-08-11 19:12:41 on dist-test-slave-t7g5"
server_key: "8a9607b443d9ae0c019c90fde11da9ff"
server_key_iv: "72b62dc3a14835a70981dc38c3dda052"
server_key_version: "encryptionkey@0"
I20260811 19:12:41.679052 20048 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260811 19:12:41.684393 20069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:41.686748 20048 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260811 19:12:41.687109 20048 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-0/wal
uuid: "025171d21e634217b58b1f0fa00e0a21"
format_stamp: "Formatted at 2026-08-11 19:12:41 on dist-test-slave-t7g5"
server_key: "8a9607b443d9ae0c019c90fde11da9ff"
server_key_iv: "72b62dc3a14835a70981dc38c3dda052"
server_key_version: "encryptionkey@0"
I20260811 19:12:41.687459 20048 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:12:41.720356 20048 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:12:41.722103 20048 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:12:41.722553 20048 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:12:41.723956 20048 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:12:41.726081 20048 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:12:41.726212 20048 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:41.726342 20048 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:12:41.726425 20048 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:41.766917 20048 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.129:40945
I20260811 19:12:41.766952 20240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.129:40945 every 8 connection(s)
I20260811 19:12:41.768976 20048 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
I20260811 19:12:41.773149 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 20048
I20260811 19:12:41.773429 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-0/wal/instance
I20260811 19:12:41.774418 3318 external_mini_cluster.cc:1428] Setting key a0bc2d9e69f384262bb6bad7cb3783d5
I20260811 19:12:41.778769 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.130:0
--local_ip_for_outbound_sockets=127.3.61.130
--webserver_interface=127.3.61.130
--webserver_port=0
--tserver_master_addrs=127.3.61.190:44367
--builtin_ntp_servers=127.3.61.148:43635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 19:12:41.788586 20245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:41.772179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:55587 (local address 127.3.61.190:44367)
0811 19:12:41.772700 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:41.772706 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:41.775238 (+ 2532us) server_negotiation.cc:407] Connection header received
0811 19:12:41.778291 (+ 3053us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:41.778298 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:41.778441 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:41.778590 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:41.780890 (+ 2300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:41.781982 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:41.784480 (+ 2498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:41.785065 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:41.785238 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:41.787461 (+ 2223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:41.787469 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:41.787473 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:12:41.787503 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:41.787564 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:12:41.787580 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:41.787674 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:41.787968 (+ 294us) server_negotiation.cc:299] Negotiation successful
0811 19:12:41.788142 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":293,"threads_started":1}
I20260811 19:12:41.790231 20246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:41.773355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:44367 (local address 127.3.61.129:55587)
0811 19:12:41.774940 (+ 1585us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:41.775009 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:41.777899 (+ 2890us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:41.778931 (+ 1032us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:41.778945 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:41.779114 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:41.780563 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:41.780600 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:41.782312 (+ 1712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:41.782317 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:41.784186 (+ 1869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:41.784215 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:41.785525 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:41.785532 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:41.785941 (+ 409us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:41.786908 (+ 967us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 19:12:41.786912 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:41.786988 (+ 76us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 19:12:41.787133 (+ 145us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:12:41.787695 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:12:41.787708 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:12:41.787718 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 19:12:41.787944 (+ 226us) client_negotiation.cc:240] Negotiation successful
0811 19:12:41.788572 (+ 628us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":830,"thread_start_us":478,"threads_started":1}
I20260811 19:12:41.792969 20241 heartbeater.cc:347] Connected to a master server at 127.3.61.190:44367
I20260811 19:12:41.793493 20241 heartbeater.cc:464] Registering TS with master...
I20260811 19:12:41.794847 20241 heartbeater.cc:511] Master 127.3.61.190:44367 requested a full tablet report, sending...
I20260811 19:12:41.797561 19943 ts_manager.cc:194] Registered new tserver with Master: 025171d21e634217b58b1f0fa00e0a21 (127.3.61.129:40945)
I20260811 19:12:41.799373 19943 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.61.129:55587
W20260811 19:12:42.202294 20247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:12:42.202677 20247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:12:42.202826 20247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:12:42.210877 20247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:12:42.211038 20247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:12:42.211179 20247 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.130
I20260811 19:12:42.222399 20247 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:43635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.61.130
--webserver_port=0
--tserver_master_addrs=127.3.61.190:44367
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.130
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:12:42.223913 20247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:12:42.226056 20247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:12:42.242002 20258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:42.242091 20256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:42.241969 20255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:12:42.243659 20247 server_base.cc:1034] running on GCE node
I20260811 19:12:42.244515 20247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:12:42.245842 20247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:12:42.247170 20247 hybrid_clock.cc:648] HybridClock initialized: now 1786475562247114 us; error 50 us; skew 500 ppm
I20260811 19:12:42.247630 20247 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 19:12:42.250613 20247 webserver.cc:460] Webserver started at http://127.3.61.130:41179/ using document root <none> and password file <none>
I20260811 19:12:42.251462 20247 fs_manager.cc:359] Metadata directory not provided
I20260811 19:12:42.251581 20247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:12:42.251935 20247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:12:42.255290 20247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-1/data/instance:
uuid: "d74d601664dc41ce902537897d1ee221"
format_stamp: "Formatted at 2026-08-11 19:12:42 on dist-test-slave-t7g5"
server_key: "92f9070f3c3e676726b4ff0c35f9bfb3"
server_key_iv: "1789659347685c8f2f9729955ff04493"
server_key_version: "encryptionkey@0"
I20260811 19:12:42.256297 20247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-1/wal/instance:
uuid: "d74d601664dc41ce902537897d1ee221"
format_stamp: "Formatted at 2026-08-11 19:12:42 on dist-test-slave-t7g5"
server_key: "92f9070f3c3e676726b4ff0c35f9bfb3"
server_key_iv: "1789659347685c8f2f9729955ff04493"
server_key_version: "encryptionkey@0"
I20260811 19:12:42.262722 20247 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260811 19:12:42.267814 20266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:42.270349 20247 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.001s
I20260811 19:12:42.270697 20247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-1/wal
uuid: "d74d601664dc41ce902537897d1ee221"
format_stamp: "Formatted at 2026-08-11 19:12:42 on dist-test-slave-t7g5"
server_key: "92f9070f3c3e676726b4ff0c35f9bfb3"
server_key_iv: "1789659347685c8f2f9729955ff04493"
server_key_version: "encryptionkey@0"
I20260811 19:12:42.271018 20247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:12:42.307307 20247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:12:42.308552 20247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:12:42.308919 20247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:12:42.310571 20247 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:12:42.312700 20247 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:12:42.312857 20247 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:42.313002 20247 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:12:42.313077 20247 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:42.353576 20247 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.130:34051
I20260811 19:12:42.353634 20431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.130:34051 every 8 connection(s)
I20260811 19:12:42.355114 20247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
I20260811 19:12:42.362282 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 20247
I20260811 19:12:42.362543 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-1/wal/instance
I20260811 19:12:42.363099 3318 external_mini_cluster.cc:1428] Setting key b8d32d2516144d4d0c9ed5261fd39599
I20260811 19:12:42.366977 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.131:0
--local_ip_for_outbound_sockets=127.3.61.131
--webserver_interface=127.3.61.131
--webserver_port=0
--tserver_master_addrs=127.3.61.190:44367
--builtin_ntp_servers=127.3.61.148:43635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 19:12:42.371398 20434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:42.358690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:36001 (local address 127.3.61.190:44367)
0811 19:12:42.360084 (+ 1394us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:42.360090 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:42.361093 (+ 1003us) server_negotiation.cc:407] Connection header received
0811 19:12:42.363267 (+ 2174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:42.363274 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:42.363387 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:42.363498 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:42.364874 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:42.365791 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:42.367604 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:42.368042 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:42.368160 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:42.370261 (+ 2101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:42.370269 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:42.370274 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:12:42.370302 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:42.370365 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:12:42.370381 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:42.370526 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:42.370792 (+ 266us) server_negotiation.cc:299] Negotiation successful
0811 19:12:42.370953 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1281,"thread_start_us":1148,"threads_started":1}
I20260811 19:12:42.373040 20436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:42.359285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:44367 (local address 127.3.61.130:36001)
0811 19:12:42.360883 (+ 1598us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:42.360929 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:42.362890 (+ 1961us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:42.363809 (+ 919us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:42.363820 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:42.363933 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:42.364649 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:42.364670 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:42.365980 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:42.365984 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:42.367424 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:42.367448 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:42.368305 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:42.368314 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:42.368576 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:42.369583 (+ 1007us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 19:12:42.369587 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:42.369688 (+ 101us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 19:12:42.369853 (+ 165us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:12:42.370505 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:12:42.370521 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:12:42.370533 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 19:12:42.370766 (+ 233us) client_negotiation.cc:240] Negotiation successful
0811 19:12:42.371437 (+ 671us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1034,"thread_start_us":781,"threads_started":1}
I20260811 19:12:42.375648 20432 heartbeater.cc:347] Connected to a master server at 127.3.61.190:44367
I20260811 19:12:42.376057 20432 heartbeater.cc:464] Registering TS with master...
I20260811 19:12:42.377234 20432 heartbeater.cc:511] Master 127.3.61.190:44367 requested a full tablet report, sending...
I20260811 19:12:42.379505 19943 ts_manager.cc:194] Registered new tserver with Master: d74d601664dc41ce902537897d1ee221 (127.3.61.130:34051)
I20260811 19:12:42.380630 19943 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.61.130:36001
W20260811 19:12:42.780932 20437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:12:42.781296 20437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:12:42.781420 20437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:12:42.789517 20437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:12:42.789701 20437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:12:42.790098 20437 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.131
I20260811 19:12:42.802997 20437 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:43635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.61.131
--webserver_port=0
--tserver_master_addrs=127.3.61.190:44367
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.131
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:12:42.804438 20241 heartbeater.cc:503] Master 127.3.61.190:44367 was elected leader, sending a full tablet report...
I20260811 19:12:42.805478 20437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:12:42.808094 20437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:12:42.827065 20446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:42.827145 20449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:42.826700 20447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:12:42.828539 20437 server_base.cc:1034] running on GCE node
I20260811 19:12:42.829365 20437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:12:42.830658 20437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:12:42.831916 20437 hybrid_clock.cc:648] HybridClock initialized: now 1786475562831850 us; error 54 us; skew 500 ppm
I20260811 19:12:42.832329 20437 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 19:12:42.836337 20437 webserver.cc:460] Webserver started at http://127.3.61.131:32999/ using document root <none> and password file <none>
I20260811 19:12:42.837220 20437 fs_manager.cc:359] Metadata directory not provided
I20260811 19:12:42.837333 20437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:12:42.837715 20437 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:12:42.842247 20437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-2/data/instance:
uuid: "fe1b1e78e8f3453cacd2720ccc78fadd"
format_stamp: "Formatted at 2026-08-11 19:12:42 on dist-test-slave-t7g5"
server_key: "611ae96ec6900c07c3f904eac98cb142"
server_key_iv: "4e7342c3a7bf3539b1722c4497f98628"
server_key_version: "encryptionkey@0"
I20260811 19:12:42.843426 20437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-2/wal/instance:
uuid: "fe1b1e78e8f3453cacd2720ccc78fadd"
format_stamp: "Formatted at 2026-08-11 19:12:42 on dist-test-slave-t7g5"
server_key: "611ae96ec6900c07c3f904eac98cb142"
server_key_iv: "4e7342c3a7bf3539b1722c4497f98628"
server_key_version: "encryptionkey@0"
I20260811 19:12:42.851220 20437 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.005s
I20260811 19:12:42.856725 20456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:42.858821 20437 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260811 19:12:42.859031 20437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-2/wal
uuid: "fe1b1e78e8f3453cacd2720ccc78fadd"
format_stamp: "Formatted at 2026-08-11 19:12:42 on dist-test-slave-t7g5"
server_key: "611ae96ec6900c07c3f904eac98cb142"
server_key_iv: "4e7342c3a7bf3539b1722c4497f98628"
server_key_version: "encryptionkey@0"
I20260811 19:12:42.859328 20437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:12:42.882841 20437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:12:42.883914 20437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:12:42.884248 20437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:12:42.885541 20437 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:12:42.887734 20437 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:12:42.887859 20437 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:42.887950 20437 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:12:42.888033 20437 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:42.926904 20437 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.131:42303
I20260811 19:12:42.926972 20634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.131:42303 every 8 connection(s)
I20260811 19:12:42.928287 20437 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
I20260811 19:12:42.930815 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 20437
I20260811 19:12:42.931028 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-2/wal/instance
I20260811 19:12:42.931561 3318 external_mini_cluster.cc:1428] Setting key 4b30c344ecba262de9d32ec0e3a69b68
I20260811 19:12:42.945386 20637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:42.931995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:34649 (local address 127.3.61.190:44367)
0811 19:12:42.932489 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:42.932496 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:42.934040 (+ 1544us) server_negotiation.cc:407] Connection header received
0811 19:12:42.936008 (+ 1968us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:42.936012 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:42.936096 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:42.936191 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:42.937888 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:42.938823 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:42.941179 (+ 2356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:42.941814 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:42.941971 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:42.944090 (+ 2119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:42.944098 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:42.944102 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:12:42.944136 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:42.944207 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:12:42.944224 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:42.944332 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:42.944753 (+ 421us) server_negotiation.cc:299] Negotiation successful
0811 19:12:42.944896 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":237,"threads_started":1}
I20260811 19:12:42.946458 20638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:42.932632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:44367 (local address 127.3.61.131:34649)
0811 19:12:42.933878 (+ 1246us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:42.933950 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:42.935691 (+ 1741us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:42.936788 (+ 1097us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:42.936799 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:42.936929 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:42.937617 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:42.937641 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:42.939086 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:42.939094 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:42.940856 (+ 1762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:42.940883 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:42.942253 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:42.942270 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:42.942608 (+ 338us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:42.943545 (+ 937us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 19:12:42.943548 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:42.943604 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 19:12:42.943721 (+ 117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:12:42.944421 (+ 700us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:12:42.944431 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:12:42.944440 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 19:12:42.944651 (+ 211us) client_negotiation.cc:240] Negotiation successful
0811 19:12:42.945064 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"thread_start_us":394,"threads_started":1}
I20260811 19:12:42.948789 20635 heartbeater.cc:347] Connected to a master server at 127.3.61.190:44367
I20260811 19:12:42.949225 20635 heartbeater.cc:464] Registering TS with master...
I20260811 19:12:42.950220 20635 heartbeater.cc:511] Master 127.3.61.190:44367 requested a full tablet report, sending...
I20260811 19:12:42.952304 19940 ts_manager.cc:194] Registered new tserver with Master: fe1b1e78e8f3453cacd2720ccc78fadd (127.3.61.131:42303)
I20260811 19:12:42.953321 19940 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.61.131:34649
I20260811 19:12:42.963151 3318 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 19:12:42.972179 20637 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:12:42.970052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35190 (local address 127.3.61.190:44367)
0811 19:12:42.970553 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:42.970561 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:42.970986 (+ 425us) server_negotiation.cc:407] Connection header received
0811 19:12:42.971190 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:42.971195 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:42.971271 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:42.971366 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:42.971904 (+ 538us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35190: BlockingRecv error: recv got EOF from 127.0.0.1:35190 (error 108)
Metrics: {"server-negotiator.queue_time_us":183}
I20260811 19:12:42.972800 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 20048
I20260811 19:12:42.983558 20649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:42.974022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49480 (local address 127.3.61.129:40945)
0811 19:12:42.974885 (+ 863us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:42.974894 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:42.974936 (+ 42us) server_negotiation.cc:407] Connection header received
0811 19:12:42.975027 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:42.975035 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:42.975362 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:42.975746 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:42.976160 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:42.977416 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:42.979340 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:42.980014 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:42.980137 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:42.981601 (+ 1464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:42.981613 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:42.981631 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:12:42.982353 (+ 722us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:42.982420 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:12:42.982448 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:42.982567 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:42.982963 (+ 396us) server_negotiation.cc:299] Negotiation successful
0811 19:12:42.983121 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626,"thread_start_us":294,"threads_started":1}
I20260811 19:12:42.985060 20232 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:12:43.072654 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 20048
I20260811 19:12:43.102298 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 20247
I20260811 19:12:43.111588 20652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:43.103560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37402 (local address 127.3.61.130:34051)
0811 19:12:43.104202 (+ 642us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:43.104211 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:43.104241 (+ 30us) server_negotiation.cc:407] Connection header received
0811 19:12:43.104321 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:43.104327 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:43.104575 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:43.104736 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:43.105225 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:43.106303 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:43.107676 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:43.108261 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:43.108390 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:43.109896 (+ 1506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:43.109910 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:43.109928 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:12:43.110616 (+ 688us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:43.110675 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:12:43.110697 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:43.110843 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:43.111020 (+ 177us) server_negotiation.cc:299] Negotiation successful
0811 19:12:43.111180 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":309,"threads_started":1}
I20260811 19:12:43.113215 20426 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:12:43.203881 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 20247
I20260811 19:12:43.235216 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 20437
I20260811 19:12:43.247460 20657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:43.236819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58472 (local address 127.3.61.131:42303)
0811 19:12:43.237859 (+ 1040us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:43.237868 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:43.237914 (+ 46us) server_negotiation.cc:407] Connection header received
0811 19:12:43.237992 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:43.237999 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:43.238260 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:43.238446 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:43.239257 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:43.240439 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:43.242334 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:43.243067 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:43.243237 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:43.244886 (+ 1649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:43.244898 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:43.244916 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:12:43.245622 (+ 706us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:43.246004 (+ 382us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:12:43.246079 (+ 75us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:43.246272 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:43.246687 (+ 415us) server_negotiation.cc:299] Negotiation successful
0811 19:12:43.246937 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":752,"thread_start_us":510,"threads_started":1}
I20260811 19:12:43.249805 20627 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:12:43.340662 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 20437
I20260811 19:12:43.372400 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 19894
I20260811 19:12:43.374154 19998 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:12:43.486506 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 19894
2026-08-11T19:12:43Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (3098 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-11T19:12:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:12:43Z Disabled control of system clock
I20260811 19:12:43.616384 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:45427
--webserver_interface=127.3.61.190
--webserver_port=0
--builtin_ntp_servers=127.3.61.148:42971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.61.190:45427
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 19:12:44.024297 20665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:12:44.024660 20665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:12:44.024763 20665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:12:44.032820 20665 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:12:44.032972 20665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:12:44.033012 20665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:12:44.033066 20665 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:12:44.033128 20665 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:12:44.043778 20665 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:42971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.61.190:45427
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:45427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.61.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:12:44.045326 20665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:12:44.047358 20665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:12:44.062932 20670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:44.066728 20671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:44.066958 20674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:12:44.067052 20665 server_base.cc:1034] running on GCE node
I20260811 19:12:44.068320 20665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:12:44.070633 20665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:12:44.072186 20665 hybrid_clock.cc:648] HybridClock initialized: now 1786475564072111 us; error 59 us; skew 500 ppm
I20260811 19:12:44.072674 20665 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 19:12:44.076087 20665 webserver.cc:460] Webserver started at http://127.3.61.190:34791/ using document root <none> and password file <none>
I20260811 19:12:44.077571 20665 fs_manager.cc:359] Metadata directory not provided
I20260811 19:12:44.077759 20665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:12:44.078119 20665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:12:44.081573 20665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/master-0/data/instance:
uuid: "71ef3d95f9134af4ae23ec4c5f3340ce"
format_stamp: "Formatted at 2026-08-11 19:12:44 on dist-test-slave-t7g5"
server_key: "47e5e5a7e8972a7a5f7978eeed963b60"
server_key_iv: "318d8cb2a318fddfac9260e811e68e85"
server_key_version: "encryptionkey@0"
I20260811 19:12:44.082626 20665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/master-0/wal/instance:
uuid: "71ef3d95f9134af4ae23ec4c5f3340ce"
format_stamp: "Formatted at 2026-08-11 19:12:44 on dist-test-slave-t7g5"
server_key: "47e5e5a7e8972a7a5f7978eeed963b60"
server_key_iv: "318d8cb2a318fddfac9260e811e68e85"
server_key_version: "encryptionkey@0"
I20260811 19:12:44.089955 20665 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260811 19:12:44.096518 20682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:44.099129 20665 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260811 19:12:44.099429 20665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/master-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/master-0/wal
uuid: "71ef3d95f9134af4ae23ec4c5f3340ce"
format_stamp: "Formatted at 2026-08-11 19:12:44 on dist-test-slave-t7g5"
server_key: "47e5e5a7e8972a7a5f7978eeed963b60"
server_key_iv: "318d8cb2a318fddfac9260e811e68e85"
server_key_version: "encryptionkey@0"
I20260811 19:12:44.099846 20665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:12:44.107331 20665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:12:44.108436 20665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:12:44.108774 20665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:12:44.133078 20665 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.190:45427
I20260811 19:12:44.133097 20770 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.190:45427 every 8 connection(s)
I20260811 19:12:44.135322 20665 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
I20260811 19:12:44.140434 20771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:12:44.145473 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 20665
I20260811 19:12:44.145852 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/master-0/wal/instance
I20260811 19:12:44.146672 3318 external_mini_cluster.cc:1428] Setting key 6dcfcf8dc2bd0050755352c4c7bc114a
I20260811 19:12:44.156203 20771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71ef3d95f9134af4ae23ec4c5f3340ce: Bootstrap starting.
I20260811 19:12:44.160151 20774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:44.149440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52746 (local address 127.3.61.190:45427)
0811 19:12:44.150922 (+ 1482us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:44.150942 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:44.151019 (+ 77us) server_negotiation.cc:407] Connection header received
0811 19:12:44.154237 (+ 3218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:44.154277 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:44.155017 (+ 740us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:44.155603 (+ 586us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:44.157487 (+ 1884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:44.157497 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:44.157517 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:12:44.157552 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:44.157649 (+ 97us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:12:44.157722 (+ 73us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:44.158021 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:44.158277 (+ 256us) server_negotiation.cc:299] Negotiation successful
0811 19:12:44.158652 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":810,"thread_start_us":525,"threads_started":1}
I20260811 19:12:44.161346 20771 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 71ef3d95f9134af4ae23ec4c5f3340ce: Neither blocks nor log segments found. Creating new log.
I20260811 19:12:44.163055 20771 log.cc:824] T 00000000000000000000000000000000 P 71ef3d95f9134af4ae23ec4c5f3340ce: Log is configured to *not* fsync() on all Append() calls
I20260811 19:12:44.167199 20771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71ef3d95f9134af4ae23ec4c5f3340ce: No bootstrap required, opened a new log
I20260811 19:12:44.176192 20771 raft_consensus.cc:348] T 00000000000000000000000000000000 P 71ef3d95f9134af4ae23ec4c5f3340ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71ef3d95f9134af4ae23ec4c5f3340ce" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 45427 } }
I20260811 19:12:44.176784 20771 raft_consensus.cc:374] T 00000000000000000000000000000000 P 71ef3d95f9134af4ae23ec4c5f3340ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:12:44.176918 20771 raft_consensus.cc:729] T 00000000000000000000000000000000 P 71ef3d95f9134af4ae23ec4c5f3340ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71ef3d95f9134af4ae23ec4c5f3340ce, State: Initialized, Role: FOLLOWER
I20260811 19:12:44.177840 20771 consensus_queue.cc:260] T 00000000000000000000000000000000 P 71ef3d95f9134af4ae23ec4c5f3340ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71ef3d95f9134af4ae23ec4c5f3340ce" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 45427 } }
I20260811 19:12:44.178128 20771 raft_consensus.cc:388] T 00000000000000000000000000000000 P 71ef3d95f9134af4ae23ec4c5f3340ce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:12:44.178241 20771 raft_consensus.cc:482] T 00000000000000000000000000000000 P 71ef3d95f9134af4ae23ec4c5f3340ce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:12:44.178395 20771 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 71ef3d95f9134af4ae23ec4c5f3340ce [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:12:44.180807 20771 raft_consensus.cc:504] T 00000000000000000000000000000000 P 71ef3d95f9134af4ae23ec4c5f3340ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71ef3d95f9134af4ae23ec4c5f3340ce" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 45427 } }
I20260811 19:12:44.181614 20771 leader_election.cc:302] T 00000000000000000000000000000000 P 71ef3d95f9134af4ae23ec4c5f3340ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 71ef3d95f9134af4ae23ec4c5f3340ce; no voters:
I20260811 19:12:44.182888 20771 leader_election.cc:288] T 00000000000000000000000000000000 P 71ef3d95f9134af4ae23ec4c5f3340ce [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:12:44.183089 20778 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 71ef3d95f9134af4ae23ec4c5f3340ce [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:12:44.184599 20778 raft_consensus.cc:686] T 00000000000000000000000000000000 P 71ef3d95f9134af4ae23ec4c5f3340ce [term 1 LEADER]: Becoming Leader. State: Replica: 71ef3d95f9134af4ae23ec4c5f3340ce, State: Running, Role: LEADER
I20260811 19:12:44.185303 20778 consensus_queue.cc:237] T 00000000000000000000000000000000 P 71ef3d95f9134af4ae23ec4c5f3340ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71ef3d95f9134af4ae23ec4c5f3340ce" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 45427 } }
I20260811 19:12:44.186278 20771 sys_catalog.cc:564] T 00000000000000000000000000000000 P 71ef3d95f9134af4ae23ec4c5f3340ce [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:12:44.189337 20779 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71ef3d95f9134af4ae23ec4c5f3340ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "71ef3d95f9134af4ae23ec4c5f3340ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71ef3d95f9134af4ae23ec4c5f3340ce" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 45427 } } }
I20260811 19:12:44.190109 20781 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71ef3d95f9134af4ae23ec4c5f3340ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 71ef3d95f9134af4ae23ec4c5f3340ce. Latest consensus state: current_term: 1 leader_uuid: "71ef3d95f9134af4ae23ec4c5f3340ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71ef3d95f9134af4ae23ec4c5f3340ce" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 45427 } } }
I20260811 19:12:44.190546 20779 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71ef3d95f9134af4ae23ec4c5f3340ce [sys.catalog]: This master's current role is: LEADER
I20260811 19:12:44.190543 20781 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71ef3d95f9134af4ae23ec4c5f3340ce [sys.catalog]: This master's current role is: LEADER
I20260811 19:12:44.196377 20790 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:12:44.206218 20790 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:12:44.220749 20790 catalog_manager.cc:1347] Generated new cluster ID: 1ae9f295e4b14a4989df37921cad69ad
I20260811 19:12:44.220897 20790 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:12:44.244100 20790 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:12:44.244902 20790 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:12:44.257166 20790 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 71ef3d95f9134af4ae23ec4c5f3340ce: Generated new TSK 0
I20260811 19:12:44.258306 20790 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:12:44.273546 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.129:0
--local_ip_for_outbound_sockets=127.3.61.129
--webserver_interface=127.3.61.129
--webserver_port=0
--tserver_master_addrs=127.3.61.190:45427
--builtin_ntp_servers=127.3.61.148:42971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 19:12:44.719684 20810 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:12:44.720058 20810 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:12:44.720165 20810 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:12:44.728006 20810 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:12:44.728144 20810 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:12:44.728266 20810 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.129
I20260811 19:12:44.742419 20810 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:42971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.61.129
--webserver_port=0
--tserver_master_addrs=127.3.61.190:45427
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.129
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:12:44.744589 20810 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:12:44.747473 20810 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:12:44.765491 20817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:44.765515 20820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:44.765488 20818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:12:44.766599 20810 server_base.cc:1034] running on GCE node
I20260811 19:12:44.767359 20810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:12:44.768512 20810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:12:44.770044 20810 hybrid_clock.cc:648] HybridClock initialized: now 1786475564769954 us; error 59 us; skew 500 ppm
I20260811 19:12:44.770524 20810 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 19:12:44.774083 20810 webserver.cc:460] Webserver started at http://127.3.61.129:40243/ using document root <none> and password file <none>
I20260811 19:12:44.775058 20810 fs_manager.cc:359] Metadata directory not provided
I20260811 19:12:44.775197 20810 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:12:44.775636 20810 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:12:44.779306 20810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-0/data/instance:
uuid: "c788bb12431448b4b5d36b52471cc666"
format_stamp: "Formatted at 2026-08-11 19:12:44 on dist-test-slave-t7g5"
server_key: "3db6237e812a4dbb15398c802fa22fe7"
server_key_iv: "bbc1103b556ae615a2c1cdb477182da3"
server_key_version: "encryptionkey@0"
I20260811 19:12:44.780411 20810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-0/wal/instance:
uuid: "c788bb12431448b4b5d36b52471cc666"
format_stamp: "Formatted at 2026-08-11 19:12:44 on dist-test-slave-t7g5"
server_key: "3db6237e812a4dbb15398c802fa22fe7"
server_key_iv: "bbc1103b556ae615a2c1cdb477182da3"
server_key_version: "encryptionkey@0"
I20260811 19:12:44.787649 20810 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.008s sys 0.000s
I20260811 19:12:44.792419 20829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:44.794675 20810 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 19:12:44.794938 20810 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-0/wal
uuid: "c788bb12431448b4b5d36b52471cc666"
format_stamp: "Formatted at 2026-08-11 19:12:44 on dist-test-slave-t7g5"
server_key: "3db6237e812a4dbb15398c802fa22fe7"
server_key_iv: "bbc1103b556ae615a2c1cdb477182da3"
server_key_version: "encryptionkey@0"
I20260811 19:12:44.795208 20810 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:12:44.802323 20810 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:12:44.803334 20810 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:12:44.803633 20810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:12:44.804855 20810 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:12:44.806736 20810 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:12:44.806864 20810 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:44.806982 20810 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:12:44.807053 20810 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:44.849099 20810 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.129:43479
I20260811 19:12:44.849216 21004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.129:43479 every 8 connection(s)
I20260811 19:12:44.850649 20810 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
I20260811 19:12:44.860963 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 20810
I20260811 19:12:44.861232 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-0/wal/instance
I20260811 19:12:44.862075 3318 external_mini_cluster.cc:1428] Setting key 179c0954ab0067913f13a6aa058805cd
I20260811 19:12:44.862112 21011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:44.854229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:34025 (local address 127.3.61.190:45427)
0811 19:12:44.854830 (+ 601us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:44.854836 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:44.856307 (+ 1471us) server_negotiation.cc:407] Connection header received
0811 19:12:44.858553 (+ 2246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:44.858559 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:44.858634 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:44.858762 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:44.861011 (+ 2249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:44.861019 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:44.861023 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:12:44.861054 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:44.861130 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:12:44.861152 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:44.861341 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:44.861643 (+ 302us) server_negotiation.cc:299] Negotiation successful
0811 19:12:44.861752 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":336,"threads_started":1}
I20260811 19:12:44.863727 21012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:44.854948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:45427 (local address 127.3.61.129:34025)
0811 19:12:44.856091 (+ 1143us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:44.856143 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:44.858126 (+ 1983us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:44.859275 (+ 1149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:44.859296 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:44.859463 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:44.860467 (+ 1004us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 19:12:44.860470 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:44.860540 (+ 70us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 19:12:44.860683 (+ 143us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:12:44.861352 (+ 669us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:12:44.861365 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:12:44.861376 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 19:12:44.861803 (+ 427us) client_negotiation.cc:240] Negotiation successful
0811 19:12:44.862151 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":359,"threads_started":1}
I20260811 19:12:44.865562 21005 heartbeater.cc:347] Connected to a master server at 127.3.61.190:45427
I20260811 19:12:44.865954 21005 heartbeater.cc:464] Registering TS with master...
I20260811 19:12:44.866228 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.130:0
--local_ip_for_outbound_sockets=127.3.61.130
--webserver_interface=127.3.61.130
--webserver_port=0
--tserver_master_addrs=127.3.61.190:45427
--builtin_ntp_servers=127.3.61.148:42971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 19:12:44.866678 21005 heartbeater.cc:511] Master 127.3.61.190:45427 requested a full tablet report, sending...
I20260811 19:12:44.869072 20707 ts_manager.cc:194] Registered new tserver with Master: c788bb12431448b4b5d36b52471cc666 (127.3.61.129:43479)
W20260811 19:12:45.298408 21013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:12:45.298777 21013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:12:45.298904 21013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:12:45.307281 21013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:12:45.307428 21013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:12:45.307586 21013 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.130
I20260811 19:12:45.318723 21013 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:42971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.61.130
--webserver_port=0
--tserver_master_addrs=127.3.61.190:45427
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.130
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:12:45.320380 21013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:12:45.322618 21013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:12:45.339004 21027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:45.339319 21030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:45.339318 21026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:12:45.340806 21013 server_base.cc:1034] running on GCE node
I20260811 19:12:45.341609 21013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:12:45.342839 21013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:12:45.344130 21013 hybrid_clock.cc:648] HybridClock initialized: now 1786475565344039 us; error 73 us; skew 500 ppm
I20260811 19:12:45.344866 21013 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 19:12:45.348757 21013 webserver.cc:460] Webserver started at http://127.3.61.130:35739/ using document root <none> and password file <none>
I20260811 19:12:45.349936 21013 fs_manager.cc:359] Metadata directory not provided
I20260811 19:12:45.350076 21013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:12:45.350472 21013 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:12:45.354156 21013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-1/data/instance:
uuid: "f662ae7cfdaa4252a35ca192834975e9"
format_stamp: "Formatted at 2026-08-11 19:12:45 on dist-test-slave-t7g5"
server_key: "4ab4130061a48171e2038a29d28755fb"
server_key_iv: "b78977aa378b6cc08028b34500bca303"
server_key_version: "encryptionkey@0"
I20260811 19:12:45.355170 21013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-1/wal/instance:
uuid: "f662ae7cfdaa4252a35ca192834975e9"
format_stamp: "Formatted at 2026-08-11 19:12:45 on dist-test-slave-t7g5"
server_key: "4ab4130061a48171e2038a29d28755fb"
server_key_iv: "b78977aa378b6cc08028b34500bca303"
server_key_version: "encryptionkey@0"
I20260811 19:12:45.362629 21013 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260811 19:12:45.367663 21039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:45.369832 21013 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.003s
I20260811 19:12:45.370078 21013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-1/wal
uuid: "f662ae7cfdaa4252a35ca192834975e9"
format_stamp: "Formatted at 2026-08-11 19:12:45 on dist-test-slave-t7g5"
server_key: "4ab4130061a48171e2038a29d28755fb"
server_key_iv: "b78977aa378b6cc08028b34500bca303"
server_key_version: "encryptionkey@0"
I20260811 19:12:45.370362 21013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:12:45.377813 21013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:12:45.378854 21013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:12:45.379160 21013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:12:45.380322 21013 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:12:45.382282 21013 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:12:45.382392 21013 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:45.382493 21013 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:12:45.382581 21013 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:45.420812 21013 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.130:43963
I20260811 19:12:45.420924 21211 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.130:43963 every 8 connection(s)
I20260811 19:12:45.422569 21013 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
I20260811 19:12:45.428558 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 21013
I20260811 19:12:45.428838 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-1/wal/instance
I20260811 19:12:45.429527 3318 external_mini_cluster.cc:1428] Setting key 609e392a4b8eab5bc829a003f8ad7fd1
I20260811 19:12:45.434906 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.131:0
--local_ip_for_outbound_sockets=127.3.61.131
--webserver_interface=127.3.61.131
--webserver_port=0
--tserver_master_addrs=127.3.61.190:45427
--builtin_ntp_servers=127.3.61.148:42971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 19:12:45.436030 21215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:45.426175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:59859 (local address 127.3.61.190:45427)
0811 19:12:45.426881 (+ 706us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:45.426886 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:45.428530 (+ 1644us) server_negotiation.cc:407] Connection header received
0811 19:12:45.431613 (+ 3083us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:45.431621 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:45.431705 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:45.431838 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:45.434745 (+ 2907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:45.434753 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:45.434757 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:12:45.434786 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:45.434937 (+ 151us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:12:45.434968 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:45.435199 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:45.435626 (+ 427us) server_negotiation.cc:299] Negotiation successful
0811 19:12:45.435706 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":414,"threads_started":1}
I20260811 19:12:45.438609 21217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:45.426682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:45427 (local address 127.3.61.130:59859)
0811 19:12:45.428208 (+ 1526us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:45.428291 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:45.431138 (+ 2847us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:45.432403 (+ 1265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:45.432424 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:45.432623 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:45.433856 (+ 1233us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 19:12:45.433860 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:45.433941 (+ 81us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 19:12:45.434378 (+ 437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:12:45.435177 (+ 799us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:12:45.435191 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:12:45.435204 (+ 13us) client_negotiation.cc:769] Sending connection context
0811 19:12:45.435740 (+ 536us) client_negotiation.cc:240] Negotiation successful
0811 19:12:45.436464 (+ 724us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":696,"thread_start_us":433,"threads_started":1}
I20260811 19:12:45.440928 21213 heartbeater.cc:347] Connected to a master server at 127.3.61.190:45427
I20260811 19:12:45.441402 21213 heartbeater.cc:464] Registering TS with master...
I20260811 19:12:45.442379 21213 heartbeater.cc:511] Master 127.3.61.190:45427 requested a full tablet report, sending...
I20260811 19:12:45.444923 20707 ts_manager.cc:194] Registered new tserver with Master: f662ae7cfdaa4252a35ca192834975e9 (127.3.61.130:43963)
W20260811 19:12:45.864944 21220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:12:45.865314 21220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:12:45.865798 21220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:12:45.874187 21220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
I20260811 19:12:45.874193 21005 heartbeater.cc:503] Master 127.3.61.190:45427 was elected leader, sending a full tablet report...
W20260811 19:12:45.874372 21220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:12:45.874547 21220 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.131
I20260811 19:12:45.885028 21220 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:42971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.61.131
--webserver_port=0
--tserver_master_addrs=127.3.61.190:45427
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.131
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:12:45.886658 21220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:12:45.888933 21220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:12:45.904893 21227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:45.904877 21229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:45.905952 21226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:12:45.908084 21220 server_base.cc:1034] running on GCE node
I20260811 19:12:45.908960 21220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:12:45.910827 21220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:12:45.912341 21220 hybrid_clock.cc:648] HybridClock initialized: now 1786475565912247 us; error 63 us; skew 500 ppm
I20260811 19:12:45.912886 21220 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 19:12:45.916143 21220 webserver.cc:460] Webserver started at http://127.3.61.131:35871/ using document root <none> and password file <none>
I20260811 19:12:45.917037 21220 fs_manager.cc:359] Metadata directory not provided
I20260811 19:12:45.917171 21220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:12:45.917557 21220 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:12:45.921094 21220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-2/data/instance:
uuid: "4714fc0db50245059d577f1e54d2c9ea"
format_stamp: "Formatted at 2026-08-11 19:12:45 on dist-test-slave-t7g5"
server_key: "9a678a5a32a1932531355e476dbdf86f"
server_key_iv: "27a95a18306576fde8858805acf4bde1"
server_key_version: "encryptionkey@0"
I20260811 19:12:45.922178 21220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-2/wal/instance:
uuid: "4714fc0db50245059d577f1e54d2c9ea"
format_stamp: "Formatted at 2026-08-11 19:12:45 on dist-test-slave-t7g5"
server_key: "9a678a5a32a1932531355e476dbdf86f"
server_key_iv: "27a95a18306576fde8858805acf4bde1"
server_key_version: "encryptionkey@0"
I20260811 19:12:45.929035 21220 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260811 19:12:45.934619 21245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:45.936959 21220 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 19:12:45.937289 21220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-2/wal
uuid: "4714fc0db50245059d577f1e54d2c9ea"
format_stamp: "Formatted at 2026-08-11 19:12:45 on dist-test-slave-t7g5"
server_key: "9a678a5a32a1932531355e476dbdf86f"
server_key_iv: "27a95a18306576fde8858805acf4bde1"
server_key_version: "encryptionkey@0"
I20260811 19:12:45.937623 21220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:12:45.945955 21220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:12:45.947063 21220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:12:45.947377 21220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:12:45.948592 21220 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:12:45.950496 21220 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:12:45.950615 21220 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:45.950748 21220 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:12:45.950821 21220 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:45.991178 21220 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.131:39181
I20260811 19:12:45.991225 21419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.131:39181 every 8 connection(s)
I20260811 19:12:45.992758 21220 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
I20260811 19:12:46.000434 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 21220
I20260811 19:12:46.000684 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786475454804547-3318-0/minicluster-data/ts-2/wal/instance
I20260811 19:12:46.001287 3318 external_mini_cluster.cc:1428] Setting key b04da070188bb90f1b1f746d4797d245
I20260811 19:12:46.008999 21424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:45.996088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:37983 (local address 127.3.61.190:45427)
0811 19:12:45.996827 (+ 739us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:45.996836 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:45.998244 (+ 1408us) server_negotiation.cc:407] Connection header received
0811 19:12:46.000528 (+ 2284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:46.000534 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:46.000632 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:46.000765 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:46.007612 (+ 6847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:46.007620 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:46.007624 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:12:46.007653 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:46.007726 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:12:46.007742 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:46.007894 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:46.008609 (+ 715us) server_negotiation.cc:299] Negotiation successful
0811 19:12:46.008688 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":399,"threads_started":1}
I20260811 19:12:46.010851 21426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:45.996568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:45427 (local address 127.3.61.131:37983)
0811 19:12:45.997990 (+ 1422us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:45.998047 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:46.000037 (+ 1990us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:46.001152 (+ 1115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:46.001164 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:46.001306 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:46.006939 (+ 5633us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 19:12:46.006944 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:46.007026 (+ 82us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 19:12:46.007220 (+ 194us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:12:46.008006 (+ 786us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:12:46.008028 (+ 22us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:12:46.008041 (+ 13us) client_negotiation.cc:769] Sending connection context
0811 19:12:46.008667 (+ 626us) client_negotiation.cc:240] Negotiation successful
0811 19:12:46.009222 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":730,"thread_start_us":442,"threads_started":1}
I20260811 19:12:46.013417 21420 heartbeater.cc:347] Connected to a master server at 127.3.61.190:45427
I20260811 19:12:46.014139 21420 heartbeater.cc:464] Registering TS with master...
I20260811 19:12:46.015105 21420 heartbeater.cc:511] Master 127.3.61.190:45427 requested a full tablet report, sending...
I20260811 19:12:46.017799 20707 ts_manager.cc:194] Registered new tserver with Master: 4714fc0db50245059d577f1e54d2c9ea (127.3.61.131:39181)
I20260811 19:12:46.021214 3318 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 19:12:46.029974 21424 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:12:46.027547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52758 (local address 127.3.61.190:45427)
0811 19:12:46.027808 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:46.027814 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:46.028721 (+ 907us) server_negotiation.cc:407] Connection header received
0811 19:12:46.028985 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:46.028989 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:46.029069 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:46.029162 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:46.029693 (+ 531us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52758: BlockingRecv error: recv got EOF from 127.0.0.1:52758 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
I20260811 19:12:46.030632 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 20810
I20260811 19:12:46.035585 21435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:46.031998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58858 (local address 127.3.61.129:43479)
0811 19:12:46.032669 (+ 671us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:46.032676 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:46.032707 (+ 31us) server_negotiation.cc:407] Connection header received
0811 19:12:46.032780 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:46.032787 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:46.033021 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:46.033170 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:46.034664 (+ 1494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:46.034673 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:46.034688 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:12:46.034705 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:46.034767 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:12:46.034785 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:46.034943 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:46.035092 (+ 149us) server_negotiation.cc:299] Negotiation successful
0811 19:12:46.035201 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":330,"threads_started":1}
I20260811 19:12:46.037865 20999 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:12:46.126503 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 20810
I20260811 19:12:46.168169 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 21013
I20260811 19:12:46.174044 21437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:46.169794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60290 (local address 127.3.61.130:43963)
0811 19:12:46.170644 (+ 850us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:46.170653 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:46.170693 (+ 40us) server_negotiation.cc:407] Connection header received
0811 19:12:46.170783 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:46.170792 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:46.171119 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:46.171293 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:46.172550 (+ 1257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:46.172560 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:46.172689 (+ 129us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:12:46.172752 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:46.172845 (+ 93us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:12:46.172884 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:46.173043 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:46.173395 (+ 352us) server_negotiation.cc:299] Negotiation successful
0811 19:12:46.173609 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":635,"thread_start_us":484,"threads_started":1}
I20260811 19:12:46.176090 21205 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:12:46.269340 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 21013
I20260811 19:12:46.300037 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 21220
I20260811 19:12:46.305253 21439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:46.301420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57014 (local address 127.3.61.131:39181)
0811 19:12:46.302217 (+ 797us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:46.302224 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:46.302261 (+ 37us) server_negotiation.cc:407] Connection header received
0811 19:12:46.302336 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:46.302344 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:46.302621 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:46.302774 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:46.304215 (+ 1441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:46.304224 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:46.304239 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:12:46.304266 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:46.304321 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:12:46.304339 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:46.304503 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:46.304737 (+ 234us) server_negotiation.cc:299] Negotiation successful
0811 19:12:46.304871 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":624,"thread_start_us":432,"threads_started":1}
I20260811 19:12:46.306833 21413 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:12:46.399149 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 21220
I20260811 19:12:46.442860 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 20665
I20260811 19:12:46.444756 20763 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:12:46.559170 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 20665
2026-08-11T19:12:46Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (3068 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:12:46 dist-test-slave-t7g5 krb5kdc[21449](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:12:46 dist-test-slave-t7g5 krb5kdc[21449](info): set up 2 sockets
Aug 11 19:12:46 dist-test-slave-t7g5 krb5kdc[21449](info): commencing operation
krb5kdc: starting...
W20260811 19:12:47.942902 3318 mini_kdc.cc:121] Time spent starting KDC: real 1.320s user 0.001s sys 0.068s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Aug 11 19:12:48 dist-test-slave-t7g5 krb5kdc[21449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:12:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:12:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.61.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab.
Entry for principal kudu/127.3.61.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab.
I20260811 19:12:48.335755 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:45273
--webserver_interface=127.3.61.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:44275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.61.190:45273
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:12:48.758867 21465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:12:48.759368 21465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:12:48.759527 21465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:12:48.768550 21465 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:12:48.768729 21465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:12:48.768786 21465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:12:48.768879 21465 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:12:48.768961 21465 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:12:48.782688 21465 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:44275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.61.190:45273
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:45273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.61.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:12:48.784771 21465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:12:48.786392 21465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:12:48.799991 21475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:48.799968 21476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:48.800192 21481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:12:48.804060 21465 server_base.cc:1034] running on GCE node
I20260811 19:12:48.805109 21465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:12:48.806915 21465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:12:48.808180 21465 hybrid_clock.cc:648] HybridClock initialized: now 1786475568808115 us; error 72 us; skew 500 ppm
I20260811 19:12:48.808588 21465 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:12:48 dist-test-slave-t7g5 krb5kdc[21449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475568, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:12:48.813134 21465 init.cc:377] Logged in from keytab as kudu/127.3.61.190@KRBTEST.COM (short username kudu)
I20260811 19:12:48.815243 21465 webserver.cc:460] Webserver started at http://127.3.61.190:42015/ using document root <none> and password file <none>
I20260811 19:12:48.816166 21465 fs_manager.cc:359] Metadata directory not provided
I20260811 19:12:48.816313 21465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:12:48.816643 21465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:12:48.820226 21465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/master-0/data/instance:
uuid: "b2c1e2dfb77a49c6a8ff944061042be9"
format_stamp: "Formatted at 2026-08-11 19:12:48 on dist-test-slave-t7g5"
server_key: "b2fb50317a422c762c3ec003be874d5b"
server_key_iv: "2eee56143aefe6c486af2449116ade65"
server_key_version: "encryptionkey@0"
I20260811 19:12:48.821143 21465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/master-0/wal/instance:
uuid: "b2c1e2dfb77a49c6a8ff944061042be9"
format_stamp: "Formatted at 2026-08-11 19:12:48 on dist-test-slave-t7g5"
server_key: "b2fb50317a422c762c3ec003be874d5b"
server_key_iv: "2eee56143aefe6c486af2449116ade65"
server_key_version: "encryptionkey@0"
I20260811 19:12:48.828621 21465 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260811 19:12:48.833108 21491 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:48.835495 21465 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 19:12:48.835855 21465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/master-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/master-0/wal
uuid: "b2c1e2dfb77a49c6a8ff944061042be9"
format_stamp: "Formatted at 2026-08-11 19:12:48 on dist-test-slave-t7g5"
server_key: "b2fb50317a422c762c3ec003be874d5b"
server_key_iv: "2eee56143aefe6c486af2449116ade65"
server_key_version: "encryptionkey@0"
I20260811 19:12:48.836164 21465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:12:48.861163 21465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:12:48.867580 21465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:12:48.867980 21465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:12:48.892611 21465 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.190:45273
I20260811 19:12:48.892633 21588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.190:45273 every 8 connection(s)
I20260811 19:12:48.894279 21465 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
I20260811 19:12:48.898860 21589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:12:48.901079 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 21465
I20260811 19:12:48.901362 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/master-0/wal/instance
I20260811 19:12:48.902257 3318 external_mini_cluster.cc:1428] Setting key 98d17a1b5068065c0614ea2994ad6771
I20260811 19:12:48.915863 21589 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2c1e2dfb77a49c6a8ff944061042be9: Bootstrap starting.
I20260811 19:12:48.920163 21589 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b2c1e2dfb77a49c6a8ff944061042be9: Neither blocks nor log segments found. Creating new log.
I20260811 19:12:48.921360 21589 log.cc:824] T 00000000000000000000000000000000 P b2c1e2dfb77a49c6a8ff944061042be9: Log is configured to *not* fsync() on all Append() calls
Aug 11 19:12:48 dist-test-slave-t7g5 krb5kdc[21449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:12:48.925269 21589 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2c1e2dfb77a49c6a8ff944061042be9: No bootstrap required, opened a new log
I20260811 19:12:48.934021 21592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:48.905076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48300 (local address 127.3.61.190:45273)
0811 19:12:48.906233 (+ 1157us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:48.906252 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:48.906817 (+ 565us) server_negotiation.cc:407] Connection header received
0811 19:12:48.910256 (+ 3439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:48.910308 (+ 52us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:48.911365 (+ 1057us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:48.912012 (+ 647us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:48.913881 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:48.916047 (+ 2166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:48.917715 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:48.918877 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:48.919119 (+ 242us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:48.924321 (+ 5202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:48.924331 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:48.924350 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:48.924400 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:48.927870 (+ 3470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:48.929206 (+ 1336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:48.929225 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:48.929237 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:48.929637 (+ 400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:48.930359 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:48.930365 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:48.930377 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:48.931182 (+ 805us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:48.931682 (+ 500us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:48.932186 (+ 504us) server_negotiation.cc:299] Negotiation successful
0811 19:12:48.932716 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":851,"thread_start_us":506,"threads_started":1}
I20260811 19:12:48.935209 21589 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2c1e2dfb77a49c6a8ff944061042be9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2c1e2dfb77a49c6a8ff944061042be9" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 45273 } }
I20260811 19:12:48.935684 21589 raft_consensus.cc:374] T 00000000000000000000000000000000 P b2c1e2dfb77a49c6a8ff944061042be9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:12:48.935809 21589 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2c1e2dfb77a49c6a8ff944061042be9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2c1e2dfb77a49c6a8ff944061042be9, State: Initialized, Role: FOLLOWER
I20260811 19:12:48.936530 21589 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2c1e2dfb77a49c6a8ff944061042be9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2c1e2dfb77a49c6a8ff944061042be9" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 45273 } }
I20260811 19:12:48.936803 21589 raft_consensus.cc:388] T 00000000000000000000000000000000 P b2c1e2dfb77a49c6a8ff944061042be9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:12:48.936955 21589 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2c1e2dfb77a49c6a8ff944061042be9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:12:48.937148 21589 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2c1e2dfb77a49c6a8ff944061042be9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:12:48.939630 21589 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2c1e2dfb77a49c6a8ff944061042be9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2c1e2dfb77a49c6a8ff944061042be9" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 45273 } }
I20260811 19:12:48.940357 21589 leader_election.cc:302] T 00000000000000000000000000000000 P b2c1e2dfb77a49c6a8ff944061042be9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b2c1e2dfb77a49c6a8ff944061042be9; no voters:
I20260811 19:12:48.941236 21589 leader_election.cc:288] T 00000000000000000000000000000000 P b2c1e2dfb77a49c6a8ff944061042be9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:12:48.941483 21595 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b2c1e2dfb77a49c6a8ff944061042be9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:12:48.942516 21595 raft_consensus.cc:686] T 00000000000000000000000000000000 P b2c1e2dfb77a49c6a8ff944061042be9 [term 1 LEADER]: Becoming Leader. State: Replica: b2c1e2dfb77a49c6a8ff944061042be9, State: Running, Role: LEADER
I20260811 19:12:48.943364 21595 consensus_queue.cc:237] T 00000000000000000000000000000000 P b2c1e2dfb77a49c6a8ff944061042be9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2c1e2dfb77a49c6a8ff944061042be9" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 45273 } }
I20260811 19:12:48.943842 21589 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2c1e2dfb77a49c6a8ff944061042be9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:12:48.946677 21598 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2c1e2dfb77a49c6a8ff944061042be9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b2c1e2dfb77a49c6a8ff944061042be9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2c1e2dfb77a49c6a8ff944061042be9" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 45273 } } }
I20260811 19:12:48.947084 21598 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2c1e2dfb77a49c6a8ff944061042be9 [sys.catalog]: This master's current role is: LEADER
I20260811 19:12:48.948463 21599 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2c1e2dfb77a49c6a8ff944061042be9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2c1e2dfb77a49c6a8ff944061042be9. Latest consensus state: current_term: 1 leader_uuid: "b2c1e2dfb77a49c6a8ff944061042be9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2c1e2dfb77a49c6a8ff944061042be9" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 45273 } } }
I20260811 19:12:48.948974 21599 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2c1e2dfb77a49c6a8ff944061042be9 [sys.catalog]: This master's current role is: LEADER
I20260811 19:12:48.951186 21603 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:12:48.957716 21603 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:12:48.970188 21603 catalog_manager.cc:1347] Generated new cluster ID: f44de2b9012741f1a79fd5fed4353dd7
I20260811 19:12:48.970715 21603 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:12:48.985311 21603 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:12:48.986871 21603 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:12:48.997051 21603 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b2c1e2dfb77a49c6a8ff944061042be9: Generated new TSK 0
I20260811 19:12:48.998422 21603 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.61.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab.
Entry for principal kudu/127.3.61.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab.
I20260811 19:12:49.093359 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.129:0
--local_ip_for_outbound_sockets=127.3.61.129
--webserver_interface=127.3.61.129
--webserver_port=0
--tserver_master_addrs=127.3.61.190:45273
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab
--principal=kudu/127.3.61.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:44275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:12:49.502642 21630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:12:49.503093 21630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:12:49.503244 21630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:12:49.512172 21630 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:12:49.512388 21630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:12:49.512547 21630 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.129
I20260811 19:12:49.524883 21630 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:44275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab
--principal=kudu/127.3.61.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.61.129
--webserver_port=0
--tserver_master_addrs=127.3.61.190:45273
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.129
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:12:49.526415 21630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:12:49.527875 21630 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:12:49.543059 21637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:49.543087 21639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:49.543015 21644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:12:49.544472 21630 server_base.cc:1034] running on GCE node
I20260811 19:12:49.545226 21630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:12:49.546389 21630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:12:49.547643 21630 hybrid_clock.cc:648] HybridClock initialized: now 1786475569547599 us; error 35 us; skew 500 ppm
I20260811 19:12:49.548000 21630 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:12:49 dist-test-slave-t7g5 krb5kdc[21449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475569, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:12:49.551983 21630 init.cc:377] Logged in from keytab as kudu/127.3.61.129@KRBTEST.COM (short username kudu)
I20260811 19:12:49.553802 21630 webserver.cc:460] Webserver started at http://127.3.61.129:42647/ using document root <none> and password file <none>
I20260811 19:12:49.554591 21630 fs_manager.cc:359] Metadata directory not provided
I20260811 19:12:49.554706 21630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:12:49.555032 21630 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:12:49.558472 21630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/ts-0/data/instance:
uuid: "73acf2a5e2e44b178dbcdceff4438c71"
format_stamp: "Formatted at 2026-08-11 19:12:49 on dist-test-slave-t7g5"
server_key: "408238bfe81927fc38e48e2836a12957"
server_key_iv: "f296f11a6a33299511e6f4bd3f76293c"
server_key_version: "encryptionkey@0"
I20260811 19:12:49.559728 21630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/ts-0/wal/instance:
uuid: "73acf2a5e2e44b178dbcdceff4438c71"
format_stamp: "Formatted at 2026-08-11 19:12:49 on dist-test-slave-t7g5"
server_key: "408238bfe81927fc38e48e2836a12957"
server_key_iv: "f296f11a6a33299511e6f4bd3f76293c"
server_key_version: "encryptionkey@0"
I20260811 19:12:49.566392 21630 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 19:12:49.570840 21654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:49.572824 21630 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260811 19:12:49.573022 21630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/ts-0/wal
uuid: "73acf2a5e2e44b178dbcdceff4438c71"
format_stamp: "Formatted at 2026-08-11 19:12:49 on dist-test-slave-t7g5"
server_key: "408238bfe81927fc38e48e2836a12957"
server_key_iv: "f296f11a6a33299511e6f4bd3f76293c"
server_key_version: "encryptionkey@0"
I20260811 19:12:49.573414 21630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:12:49.590934 21630 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:12:49.597047 21630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:12:49.597461 21630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:12:49.599025 21630 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:12:49.600668 21630 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:12:49.600759 21630 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:49.600878 21630 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:12:49.600948 21630 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:49.640537 21630 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.129:35043
I20260811 19:12:49.640547 21833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.129:35043 every 8 connection(s)
I20260811 19:12:49.642102 21630 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
I20260811 19:12:49.646942 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 21630
I20260811 19:12:49.647267 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/ts-0/wal/instance
I20260811 19:12:49.648281 3318 external_mini_cluster.cc:1428] Setting key 6aa81295c2330dd612cea4021c8b037d
Aug 11 19:12:49 dist-test-slave-t7g5 krb5kdc[21449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475569, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.129@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:12:49.669929 21837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:49.645022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:48297 (local address 127.3.61.190:45273)
0811 19:12:49.645791 (+ 769us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:49.645796 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:49.647343 (+ 1547us) server_negotiation.cc:407] Connection header received
0811 19:12:49.650719 (+ 3376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:49.650728 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:49.650851 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:49.651021 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:49.653931 (+ 2910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:49.654852 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:49.657281 (+ 2429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:49.657854 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:49.657995 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:49.662949 (+ 4954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:49.662957 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:49.662960 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:49.663009 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:49.665709 (+ 2700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:49.666784 (+ 1075us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:49.666790 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:49.666793 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:49.666966 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:49.667425 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:49.667430 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:49.667433 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:49.667826 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:49.667998 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:49.669243 (+ 1245us) server_negotiation.cc:299] Negotiation successful
0811 19:12:49.669536 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":384,"threads_started":1}
I20260811 19:12:49.671355 21838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:49.645752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:45273 (local address 127.3.61.129:48297)
0811 19:12:49.647049 (+ 1297us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:49.647149 (+ 100us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:49.650127 (+ 2978us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:49.651436 (+ 1309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:49.651449 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:49.652643 (+ 1194us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:49.653602 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:49.653635 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:49.655075 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:49.655081 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:49.657044 (+ 1963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:49.657066 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:49.658143 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:49.658152 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:49.658460 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:49.659386 (+ 926us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:49.659418 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:49.662541 (+ 3123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:12:49.665929 (+ 3388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:49.665940 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:49.665962 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:49.666608 (+ 646us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:49.667088 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:49.667092 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:49.667095 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:49.667311 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:49.668078 (+ 767us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:12:49.668090 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:12:49.668695 (+ 605us) client_negotiation.cc:769] Sending connection context
0811 19:12:49.669059 (+ 364us) client_negotiation.cc:240] Negotiation successful
0811 19:12:49.669801 (+ 742us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":845,"thread_start_us":500,"threads_started":1}
I20260811 19:12:49.673705 21834 heartbeater.cc:347] Connected to a master server at 127.3.61.190:45273
I20260811 19:12:49.674101 21834 heartbeater.cc:464] Registering TS with master...
I20260811 19:12:49.674998 21834 heartbeater.cc:511] Master 127.3.61.190:45273 requested a full tablet report, sending...
I20260811 19:12:49.677902 21523 ts_manager.cc:194] Registered new tserver with Master: 73acf2a5e2e44b178dbcdceff4438c71 (127.3.61.129:35043)
I20260811 19:12:49.679777 21523 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.129@KRBTEST.COM'} at 127.3.61.129:48297
WARNING: no policy specified for kudu/127.3.61.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab.
Entry for principal kudu/127.3.61.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab.
I20260811 19:12:49.727797 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.130:0
--local_ip_for_outbound_sockets=127.3.61.130
--webserver_interface=127.3.61.130
--webserver_port=0
--tserver_master_addrs=127.3.61.190:45273
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab
--principal=kudu/127.3.61.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:44275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:12:50.134002 21842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:12:50.134369 21842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:12:50.134521 21842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:12:50.142895 21842 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:12:50.143067 21842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:12:50.143229 21842 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.130
I20260811 19:12:50.157760 21842 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:44275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab
--principal=kudu/127.3.61.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.61.130
--webserver_port=0
--tserver_master_addrs=127.3.61.190:45273
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.130
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:12:50.159608 21842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:12:50.161170 21842 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:12:50.178715 21854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:50.179397 21856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:50.179432 21852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:12:50.180285 21842 server_base.cc:1034] running on GCE node
I20260811 19:12:50.181048 21842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:12:50.182263 21842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:12:50.183540 21842 hybrid_clock.cc:648] HybridClock initialized: now 1786475570183501 us; error 41 us; skew 500 ppm
I20260811 19:12:50.183939 21842 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:12:50 dist-test-slave-t7g5 krb5kdc[21449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475570, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:12:50.188359 21842 init.cc:377] Logged in from keytab as kudu/127.3.61.130@KRBTEST.COM (short username kudu)
I20260811 19:12:50.193745 21842 webserver.cc:460] Webserver started at http://127.3.61.130:46229/ using document root <none> and password file <none>
I20260811 19:12:50.195013 21842 fs_manager.cc:359] Metadata directory not provided
I20260811 19:12:50.195215 21842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:12:50.195680 21842 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:12:50.200389 21842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/ts-1/data/instance:
uuid: "975ccb3c32144fdb8c932958cce7c9e0"
format_stamp: "Formatted at 2026-08-11 19:12:50 on dist-test-slave-t7g5"
server_key: "da9a1d3dae3f5f4a2dad25b7fcd0714b"
server_key_iv: "61784aff007f8a7b36fd0ca482f2ba44"
server_key_version: "encryptionkey@0"
I20260811 19:12:50.201366 21842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/ts-1/wal/instance:
uuid: "975ccb3c32144fdb8c932958cce7c9e0"
format_stamp: "Formatted at 2026-08-11 19:12:50 on dist-test-slave-t7g5"
server_key: "da9a1d3dae3f5f4a2dad25b7fcd0714b"
server_key_iv: "61784aff007f8a7b36fd0ca482f2ba44"
server_key_version: "encryptionkey@0"
I20260811 19:12:50.208510 21842 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260811 19:12:50.213824 21866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:50.216315 21842 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260811 19:12:50.216600 21842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/ts-1/wal
uuid: "975ccb3c32144fdb8c932958cce7c9e0"
format_stamp: "Formatted at 2026-08-11 19:12:50 on dist-test-slave-t7g5"
server_key: "da9a1d3dae3f5f4a2dad25b7fcd0714b"
server_key_iv: "61784aff007f8a7b36fd0ca482f2ba44"
server_key_version: "encryptionkey@0"
I20260811 19:12:50.216909 21842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:12:50.254479 21842 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:12:50.260998 21842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:12:50.261376 21842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:12:50.262687 21842 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:12:50.264307 21842 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:12:50.264411 21842 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:50.264529 21842 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:12:50.264598 21842 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:50.303887 21842 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.130:42025
I20260811 19:12:50.303925 22046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.130:42025 every 8 connection(s)
I20260811 19:12:50.305393 21842 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
I20260811 19:12:50.310770 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 21842
I20260811 19:12:50.311110 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/ts-1/wal/instance
I20260811 19:12:50.311776 3318 external_mini_cluster.cc:1428] Setting key f0b037178415756007870f9dd6fa5b61
Aug 11 19:12:50 dist-test-slave-t7g5 krb5kdc[21449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475570, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.130@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:12:50.336860 22050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:50.309237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:41031 (local address 127.3.61.190:45273)
0811 19:12:50.309911 (+ 674us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:50.309920 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:50.311458 (+ 1538us) server_negotiation.cc:407] Connection header received
0811 19:12:50.313955 (+ 2497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:50.313961 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:50.314076 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:50.314357 (+ 281us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:50.317061 (+ 2704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:50.318152 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:50.320927 (+ 2775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:50.321490 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:50.321700 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:50.328054 (+ 6354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:50.328064 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:50.328069 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:50.328126 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:50.331474 (+ 3348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:50.333025 (+ 1551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:50.333032 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:50.333035 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:50.333380 (+ 345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:50.334309 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:50.334315 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:50.334317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:50.334722 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:50.334888 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:50.336286 (+ 1398us) server_negotiation.cc:299] Negotiation successful
0811 19:12:50.336563 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":443,"threads_started":1}
I20260811 19:12:50.338807 22051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:50.309983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:45273 (local address 127.3.61.130:41031)
0811 19:12:50.311179 (+ 1196us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:50.311240 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:50.313549 (+ 2309us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:50.314682 (+ 1133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:50.314693 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:50.315642 (+ 949us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:50.316728 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:50.316764 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:50.318418 (+ 1654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:50.318424 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:50.320594 (+ 2170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:50.320630 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:50.321943 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:50.321954 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:50.322456 (+ 502us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:50.323585 (+ 1129us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:50.323636 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:50.327470 (+ 3834us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:12:50.331737 (+ 4267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:50.331749 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:50.331773 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:50.332734 (+ 961us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:50.333555 (+ 821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:50.333561 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:50.333565 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:50.333988 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:50.334922 (+ 934us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:12:50.334935 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:12:50.335651 (+ 716us) client_negotiation.cc:769] Sending connection context
0811 19:12:50.336097 (+ 446us) client_negotiation.cc:240] Negotiation successful
0811 19:12:50.336890 (+ 793us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":827,"thread_start_us":452,"threads_started":1}
I20260811 19:12:50.341199 22048 heartbeater.cc:347] Connected to a master server at 127.3.61.190:45273
I20260811 19:12:50.341697 22048 heartbeater.cc:464] Registering TS with master...
I20260811 19:12:50.342830 22048 heartbeater.cc:511] Master 127.3.61.190:45273 requested a full tablet report, sending...
I20260811 19:12:50.345163 21523 ts_manager.cc:194] Registered new tserver with Master: 975ccb3c32144fdb8c932958cce7c9e0 (127.3.61.130:42025)
I20260811 19:12:50.346618 21523 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.130@KRBTEST.COM'} at 127.3.61.130:41031
WARNING: no policy specified for kudu/127.3.61.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab.
Entry for principal kudu/127.3.61.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab.
I20260811 19:12:50.406520 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.131:0
--local_ip_for_outbound_sockets=127.3.61.131
--webserver_interface=127.3.61.131
--webserver_port=0
--tserver_master_addrs=127.3.61.190:45273
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab
--principal=kudu/127.3.61.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:44275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:12:50.684399 21834 heartbeater.cc:503] Master 127.3.61.190:45273 was elected leader, sending a full tablet report...
W20260811 19:12:50.814128 22054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:12:50.814465 22054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:12:50.814611 22054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:12:50.822651 22054 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:12:50.822809 22054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:12:50.822989 22054 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.131
I20260811 19:12:50.834329 22054 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:44275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab
--principal=kudu/127.3.61.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.61.131
--webserver_port=0
--tserver_master_addrs=127.3.61.190:45273
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.131
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:12:50.836014 22054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:12:50.837841 22054 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:12:50.855295 22067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:50.855252 22063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:12:50.856216 22062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:12:50.856247 22054 server_base.cc:1034] running on GCE node
I20260811 19:12:50.857062 22054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:12:50.858698 22054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:12:50.860013 22054 hybrid_clock.cc:648] HybridClock initialized: now 1786475570859953 us; error 44 us; skew 500 ppm
I20260811 19:12:50.860457 22054 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:12:50 dist-test-slave-t7g5 krb5kdc[21449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475570, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:12:50.864354 22054 init.cc:377] Logged in from keytab as kudu/127.3.61.131@KRBTEST.COM (short username kudu)
I20260811 19:12:50.866304 22054 webserver.cc:460] Webserver started at http://127.3.61.131:35579/ using document root <none> and password file <none>
I20260811 19:12:50.867094 22054 fs_manager.cc:359] Metadata directory not provided
I20260811 19:12:50.867235 22054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:12:50.867571 22054 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:12:50.871003 22054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/ts-2/data/instance:
uuid: "167fbdde16bc424a8961aab5463ba4df"
format_stamp: "Formatted at 2026-08-11 19:12:50 on dist-test-slave-t7g5"
server_key: "a9a9063d4174ab4e71961090f3e1c4d3"
server_key_iv: "f89f87af3ba23c186a8dff15c1bf5c3d"
server_key_version: "encryptionkey@0"
I20260811 19:12:50.871918 22054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/ts-2/wal/instance:
uuid: "167fbdde16bc424a8961aab5463ba4df"
format_stamp: "Formatted at 2026-08-11 19:12:50 on dist-test-slave-t7g5"
server_key: "a9a9063d4174ab4e71961090f3e1c4d3"
server_key_iv: "f89f87af3ba23c186a8dff15c1bf5c3d"
server_key_version: "encryptionkey@0"
I20260811 19:12:50.879302 22054 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.004s
I20260811 19:12:50.884755 22082 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:50.886777 22054 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260811 19:12:50.887022 22054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/ts-2/wal
uuid: "167fbdde16bc424a8961aab5463ba4df"
format_stamp: "Formatted at 2026-08-11 19:12:50 on dist-test-slave-t7g5"
server_key: "a9a9063d4174ab4e71961090f3e1c4d3"
server_key_iv: "f89f87af3ba23c186a8dff15c1bf5c3d"
server_key_version: "encryptionkey@0"
I20260811 19:12:50.887281 22054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:12:50.908180 22054 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:12:50.914350 22054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:12:50.914763 22054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:12:50.916205 22054 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:12:50.918588 22054 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:12:50.918696 22054 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:50.918850 22054 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:12:50.918911 22054 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:12:50.958856 22054 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.131:43187
I20260811 19:12:50.958902 22251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.131:43187 every 8 connection(s)
I20260811 19:12:50.960306 22054 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
I20260811 19:12:50.966068 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 22054
I20260811 19:12:50.966341 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/ts-2/wal/instance
I20260811 19:12:50.966885 3318 external_mini_cluster.cc:1428] Setting key 83832c176b5e81645bbc3abad9cbeef9
Aug 11 19:12:50 dist-test-slave-t7g5 krb5kdc[21449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475570, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.131@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:12:50.984754 22255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:50.963379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:45341 (local address 127.3.61.190:45273)
0811 19:12:50.963854 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:50.963858 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:50.965138 (+ 1280us) server_negotiation.cc:407] Connection header received
0811 19:12:50.967285 (+ 2147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:50.967291 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:50.967407 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:50.967508 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:50.969550 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:50.970484 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:50.972746 (+ 2262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:50.973197 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:50.973322 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:50.977446 (+ 4124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:50.977451 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:50.977455 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:50.977498 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:50.980435 (+ 2937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:50.981326 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:50.981333 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:50.981337 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:50.981503 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:50.982221 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:50.982228 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:50.982232 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:50.982783 (+ 551us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:50.983009 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:50.984058 (+ 1049us) server_negotiation.cc:299] Negotiation successful
0811 19:12:50.984388 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":228,"threads_started":1}
I20260811 19:12:50.985617 22256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:50.963820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:45273 (local address 127.3.61.131:45341)
0811 19:12:50.964937 (+ 1117us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:50.964986 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:50.966937 (+ 1951us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:50.967821 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:50.967831 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:50.968596 (+ 765us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:12:50.969339 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:50.969370 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:50.970643 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:50.970647 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:50.972429 (+ 1782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:50.972461 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:50.973490 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:50.973500 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:50.973932 (+ 432us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:50.974657 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:12:50.974683 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:12:50.977149 (+ 2466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:12:50.980634 (+ 3485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:50.980646 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:50.980665 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:50.981180 (+ 515us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:50.981693 (+ 513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:12:50.981698 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:12:50.981702 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:12:50.982004 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:12:50.982975 (+ 971us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:12:50.982984 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:12:50.983536 (+ 552us) client_negotiation.cc:769] Sending connection context
0811 19:12:50.983864 (+ 328us) client_negotiation.cc:240] Negotiation successful
0811 19:12:50.984361 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":844,"thread_start_us":459,"threads_started":1}
I20260811 19:12:50.987413 22252 heartbeater.cc:347] Connected to a master server at 127.3.61.190:45273
I20260811 19:12:50.987775 22252 heartbeater.cc:464] Registering TS with master...
I20260811 19:12:50.988693 22252 heartbeater.cc:511] Master 127.3.61.190:45273 requested a full tablet report, sending...
I20260811 19:12:50.991225 21523 ts_manager.cc:194] Registered new tserver with Master: 167fbdde16bc424a8961aab5463ba4df (127.3.61.131:43187)
I20260811 19:12:50.992228 21523 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.131@KRBTEST.COM'} at 127.3.61.131:45341
I20260811 19:12:51.002422 3318 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 19:12:51.026158 22255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:51.007767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48316 (local address 127.3.61.190:45273)
0811 19:12:51.007959 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:51.007964 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:51.008457 (+ 493us) server_negotiation.cc:407] Connection header received
0811 19:12:51.008572 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:51.008576 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:51.008650 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:51.008751 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:51.010578 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:51.011513 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:51.013029 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:51.013553 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:51.013719 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:51.019172 (+ 5453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:51.019178 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:51.019182 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:51.019227 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:51.022309 (+ 3082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:51.023544 (+ 1235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:51.023548 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:51.023551 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:51.023717 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:51.024346 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:51.024350 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:51.024352 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:51.024745 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:51.024975 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:51.025600 (+ 625us) server_negotiation.cc:299] Negotiation successful
0811 19:12:51.025899 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 19:12:51.032635 21523 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48316:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 19:12:51.041121 21523 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:12:51.085156 22271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:51.059258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39174 (local address 127.3.61.129:35043)
0811 19:12:51.070902 (+ 11644us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:51.070914 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:51.072240 (+ 1326us) server_negotiation.cc:407] Connection header received
0811 19:12:51.072735 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:51.072746 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:51.073032 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:51.073220 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:12:51.074751 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:51.076981 (+ 2230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:51.082182 (+ 5201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:51.083782 (+ 1600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:51.083959 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:51.084235 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:51.084584 (+ 349us) server_negotiation.cc:299] Negotiation successful
0811 19:12:51.084785 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11377,"thread_start_us":11665,"threads_started":1}
I20260811 19:12:51.085167 22270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:51.059083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:35043 (local address 127.0.0.1:39174)
0811 19:12:51.072029 (+ 12946us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:51.072093 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:51.072456 (+ 363us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:51.073644 (+ 1188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:51.073678 (+ 34us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:51.073726 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:12:51.074461 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:51.074478 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:51.077251 (+ 2773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:51.077258 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:51.081931 (+ 4673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:51.081959 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:51.084030 (+ 2071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:51.084037 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:51.084258 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:51.084328 (+ 70us) client_negotiation.cc:769] Sending connection context
0811 19:12:51.084482 (+ 154us) client_negotiation.cc:240] Negotiation successful
0811 19:12:51.084705 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12641,"mutex_wait_us":638,"spinlock_wait_cycles":21632,"thread_start_us":12294,"threads_started":1}
I20260811 19:12:51.085337 22275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:51.060164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38266 (local address 127.3.61.131:43187)
0811 19:12:51.070799 (+ 10635us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:51.070812 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:51.072239 (+ 1427us) server_negotiation.cc:407] Connection header received
0811 19:12:51.072848 (+ 609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:51.072857 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:51.073096 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:51.073272 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:12:51.074750 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:51.077288 (+ 2538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:51.082511 (+ 5223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:51.083965 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:51.084087 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:51.084281 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:51.084801 (+ 520us) server_negotiation.cc:299] Negotiation successful
0811 19:12:51.084992 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10350,"thread_start_us":10619,"threads_started":1}
I20260811 19:12:51.086278 22274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:51.059382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.131:43187 (local address 127.0.0.1:38266)
0811 19:12:51.072029 (+ 12647us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:51.072094 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:51.072453 (+ 359us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:51.073642 (+ 1189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:51.073678 (+ 36us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:51.073728 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:12:51.074472 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:51.074487 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:51.078859 (+ 4372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:51.078866 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:51.082284 (+ 3418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:51.082301 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:51.084403 (+ 2102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:51.084408 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:51.084581 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:51.084615 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 19:12:51.085625 (+ 1010us) client_negotiation.cc:240] Negotiation successful
0811 19:12:51.085823 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12429,"mutex_wait_us":96,"spinlock_wait_cycles":13440,"thread_start_us":11968,"threads_started":1}
I20260811 19:12:51.090322 22276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:51.072866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49540 (local address 127.3.61.130:42025)
0811 19:12:51.081707 (+ 8841us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:51.081719 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:51.082407 (+ 688us) server_negotiation.cc:407] Connection header received
0811 19:12:51.082530 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:51.082538 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:51.082864 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:51.083102 (+ 238us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:12:51.083812 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:51.085310 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:51.087814 (+ 2504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:51.088802 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:51.088941 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:51.089163 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:51.089788 (+ 625us) server_negotiation.cc:299] Negotiation successful
0811 19:12:51.089979 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8628,"thread_start_us":8465,"threads_started":1}
I20260811 19:12:51.090454 22277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:51.073929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.130:42025 (local address 127.0.0.1:49540)
0811 19:12:51.082094 (+ 8165us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:51.082114 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:51.082272 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:51.083237 (+ 965us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:51.083242 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:51.083282 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:12:51.083612 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:51.083621 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:51.085498 (+ 1877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:51.085502 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:51.087633 (+ 2131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:51.087650 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:51.089033 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:51.089041 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:51.089223 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:51.089257 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 19:12:51.089350 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 19:12:51.089491 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8033,"thread_start_us":7873,"threads_started":1}
I20260811 19:12:51.090257 22151 tablet_service.cc:1467] Processing CreateTablet for tablet 4bfdf998f23d492fb8e7c559fede96bb (DEFAULT_TABLE table=test-table [id=01b850beacec407f926097378a995207]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:12:51.090257 21734 tablet_service.cc:1467] Processing CreateTablet for tablet 4bfdf998f23d492fb8e7c559fede96bb (DEFAULT_TABLE table=test-table [id=01b850beacec407f926097378a995207]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:12:51.092336 21734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4bfdf998f23d492fb8e7c559fede96bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:12:51.092334 22151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4bfdf998f23d492fb8e7c559fede96bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:12:51.094295 21944 tablet_service.cc:1467] Processing CreateTablet for tablet 4bfdf998f23d492fb8e7c559fede96bb (DEFAULT_TABLE table=test-table [id=01b850beacec407f926097378a995207]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:12:51.095746 21944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4bfdf998f23d492fb8e7c559fede96bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:12:51.109318 22280 tablet_bootstrap.cc:492] T 4bfdf998f23d492fb8e7c559fede96bb P 167fbdde16bc424a8961aab5463ba4df: Bootstrap starting.
I20260811 19:12:51.110374 22279 tablet_bootstrap.cc:492] T 4bfdf998f23d492fb8e7c559fede96bb P 73acf2a5e2e44b178dbcdceff4438c71: Bootstrap starting.
I20260811 19:12:51.111961 22281 tablet_bootstrap.cc:492] T 4bfdf998f23d492fb8e7c559fede96bb P 975ccb3c32144fdb8c932958cce7c9e0: Bootstrap starting.
I20260811 19:12:51.115448 22279 tablet_bootstrap.cc:654] T 4bfdf998f23d492fb8e7c559fede96bb P 73acf2a5e2e44b178dbcdceff4438c71: Neither blocks nor log segments found. Creating new log.
I20260811 19:12:51.115655 22281 tablet_bootstrap.cc:654] T 4bfdf998f23d492fb8e7c559fede96bb P 975ccb3c32144fdb8c932958cce7c9e0: Neither blocks nor log segments found. Creating new log.
I20260811 19:12:51.115949 22280 tablet_bootstrap.cc:654] T 4bfdf998f23d492fb8e7c559fede96bb P 167fbdde16bc424a8961aab5463ba4df: Neither blocks nor log segments found. Creating new log.
I20260811 19:12:51.116837 22281 log.cc:824] T 4bfdf998f23d492fb8e7c559fede96bb P 975ccb3c32144fdb8c932958cce7c9e0: Log is configured to *not* fsync() on all Append() calls
I20260811 19:12:51.116931 22279 log.cc:824] T 4bfdf998f23d492fb8e7c559fede96bb P 73acf2a5e2e44b178dbcdceff4438c71: Log is configured to *not* fsync() on all Append() calls
I20260811 19:12:51.117192 22280 log.cc:824] T 4bfdf998f23d492fb8e7c559fede96bb P 167fbdde16bc424a8961aab5463ba4df: Log is configured to *not* fsync() on all Append() calls
I20260811 19:12:51.120747 22279 tablet_bootstrap.cc:492] T 4bfdf998f23d492fb8e7c559fede96bb P 73acf2a5e2e44b178dbcdceff4438c71: No bootstrap required, opened a new log
I20260811 19:12:51.121115 22279 ts_tablet_manager.cc:1397] T 4bfdf998f23d492fb8e7c559fede96bb P 73acf2a5e2e44b178dbcdceff4438c71: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.002s
I20260811 19:12:51.122076 22280 tablet_bootstrap.cc:492] T 4bfdf998f23d492fb8e7c559fede96bb P 167fbdde16bc424a8961aab5463ba4df: No bootstrap required, opened a new log
I20260811 19:12:51.122076 22281 tablet_bootstrap.cc:492] T 4bfdf998f23d492fb8e7c559fede96bb P 975ccb3c32144fdb8c932958cce7c9e0: No bootstrap required, opened a new log
I20260811 19:12:51.122517 22281 ts_tablet_manager.cc:1397] T 4bfdf998f23d492fb8e7c559fede96bb P 975ccb3c32144fdb8c932958cce7c9e0: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260811 19:12:51.122514 22280 ts_tablet_manager.cc:1397] T 4bfdf998f23d492fb8e7c559fede96bb P 167fbdde16bc424a8961aab5463ba4df: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.003s
I20260811 19:12:51.131174 22279 raft_consensus.cc:348] T 4bfdf998f23d492fb8e7c559fede96bb P 73acf2a5e2e44b178dbcdceff4438c71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73acf2a5e2e44b178dbcdceff4438c71" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 35043 } } peers { permanent_uuid: "975ccb3c32144fdb8c932958cce7c9e0" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 42025 } } peers { permanent_uuid: "167fbdde16bc424a8961aab5463ba4df" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 43187 } }
I20260811 19:12:51.131690 22279 raft_consensus.cc:374] T 4bfdf998f23d492fb8e7c559fede96bb P 73acf2a5e2e44b178dbcdceff4438c71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:12:51.131812 22279 raft_consensus.cc:729] T 4bfdf998f23d492fb8e7c559fede96bb P 73acf2a5e2e44b178dbcdceff4438c71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73acf2a5e2e44b178dbcdceff4438c71, State: Initialized, Role: FOLLOWER
I20260811 19:12:51.132699 22279 consensus_queue.cc:260] T 4bfdf998f23d492fb8e7c559fede96bb P 73acf2a5e2e44b178dbcdceff4438c71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73acf2a5e2e44b178dbcdceff4438c71" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 35043 } } peers { permanent_uuid: "975ccb3c32144fdb8c932958cce7c9e0" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 42025 } } peers { permanent_uuid: "167fbdde16bc424a8961aab5463ba4df" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 43187 } }
I20260811 19:12:51.134865 22279 ts_tablet_manager.cc:1428] T 4bfdf998f23d492fb8e7c559fede96bb P 73acf2a5e2e44b178dbcdceff4438c71: Time spent starting tablet: real 0.014s user 0.006s sys 0.009s
I20260811 19:12:51.135016 22281 raft_consensus.cc:348] T 4bfdf998f23d492fb8e7c559fede96bb P 975ccb3c32144fdb8c932958cce7c9e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73acf2a5e2e44b178dbcdceff4438c71" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 35043 } } peers { permanent_uuid: "975ccb3c32144fdb8c932958cce7c9e0" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 42025 } } peers { permanent_uuid: "167fbdde16bc424a8961aab5463ba4df" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 43187 } }
I20260811 19:12:51.134999 22280 raft_consensus.cc:348] T 4bfdf998f23d492fb8e7c559fede96bb P 167fbdde16bc424a8961aab5463ba4df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73acf2a5e2e44b178dbcdceff4438c71" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 35043 } } peers { permanent_uuid: "975ccb3c32144fdb8c932958cce7c9e0" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 42025 } } peers { permanent_uuid: "167fbdde16bc424a8961aab5463ba4df" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 43187 } }
I20260811 19:12:51.135565 22281 raft_consensus.cc:374] T 4bfdf998f23d492fb8e7c559fede96bb P 975ccb3c32144fdb8c932958cce7c9e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:12:51.135561 22280 raft_consensus.cc:374] T 4bfdf998f23d492fb8e7c559fede96bb P 167fbdde16bc424a8961aab5463ba4df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:12:51.135694 22281 raft_consensus.cc:729] T 4bfdf998f23d492fb8e7c559fede96bb P 975ccb3c32144fdb8c932958cce7c9e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 975ccb3c32144fdb8c932958cce7c9e0, State: Initialized, Role: FOLLOWER
I20260811 19:12:51.135689 22280 raft_consensus.cc:729] T 4bfdf998f23d492fb8e7c559fede96bb P 167fbdde16bc424a8961aab5463ba4df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 167fbdde16bc424a8961aab5463ba4df, State: Initialized, Role: FOLLOWER
I20260811 19:12:51.136448 22280 consensus_queue.cc:260] T 4bfdf998f23d492fb8e7c559fede96bb P 167fbdde16bc424a8961aab5463ba4df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73acf2a5e2e44b178dbcdceff4438c71" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 35043 } } peers { permanent_uuid: "975ccb3c32144fdb8c932958cce7c9e0" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 42025 } } peers { permanent_uuid: "167fbdde16bc424a8961aab5463ba4df" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 43187 } }
I20260811 19:12:51.136446 22281 consensus_queue.cc:260] T 4bfdf998f23d492fb8e7c559fede96bb P 975ccb3c32144fdb8c932958cce7c9e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73acf2a5e2e44b178dbcdceff4438c71" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 35043 } } peers { permanent_uuid: "975ccb3c32144fdb8c932958cce7c9e0" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 42025 } } peers { permanent_uuid: "167fbdde16bc424a8961aab5463ba4df" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 43187 } }
I20260811 19:12:51.138577 22280 ts_tablet_manager.cc:1428] T 4bfdf998f23d492fb8e7c559fede96bb P 167fbdde16bc424a8961aab5463ba4df: Time spent starting tablet: real 0.016s user 0.015s sys 0.001s
I20260811 19:12:51.139207 22252 heartbeater.cc:503] Master 127.3.61.190:45273 was elected leader, sending a full tablet report...
I20260811 19:12:51.139286 22048 heartbeater.cc:503] Master 127.3.61.190:45273 was elected leader, sending a full tablet report...
I20260811 19:12:51.139616 22281 ts_tablet_manager.cc:1428] T 4bfdf998f23d492fb8e7c559fede96bb P 975ccb3c32144fdb8c932958cce7c9e0: Time spent starting tablet: real 0.017s user 0.008s sys 0.008s
I20260811 19:12:51.144794 22289 raft_consensus.cc:482] T 4bfdf998f23d492fb8e7c559fede96bb P 167fbdde16bc424a8961aab5463ba4df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260811 19:12:51.144956 21835 tablet.cc:2369] T 4bfdf998f23d492fb8e7c559fede96bb P 73acf2a5e2e44b178dbcdceff4438c71: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:12:51.145135 22289 raft_consensus.cc:504] T 4bfdf998f23d492fb8e7c559fede96bb P 167fbdde16bc424a8961aab5463ba4df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73acf2a5e2e44b178dbcdceff4438c71" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 35043 } } peers { permanent_uuid: "975ccb3c32144fdb8c932958cce7c9e0" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 42025 } } peers { permanent_uuid: "167fbdde16bc424a8961aab5463ba4df" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 43187 } }
I20260811 19:12:51.147457 22289 leader_election.cc:288] T 4bfdf998f23d492fb8e7c559fede96bb P 167fbdde16bc424a8961aab5463ba4df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73acf2a5e2e44b178dbcdceff4438c71 (127.3.61.129:35043), 975ccb3c32144fdb8c932958cce7c9e0 (127.3.61.130:42025)
I20260811 19:12:51.154825 22271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:51.148892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:37977 (local address 127.3.61.129:35043)
0811 19:12:51.149157 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:51.149172 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:51.149195 (+ 23us) server_negotiation.cc:407] Connection header received
0811 19:12:51.149290 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:51.149294 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:51.149377 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:51.149604 (+ 227us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:12:51.150315 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:51.151107 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:51.152990 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:51.153904 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:51.154068 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:51.154152 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:51.154423 (+ 271us) server_negotiation.cc:299] Negotiation successful
0811 19:12:51.154565 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 19:12:51.154871 22256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:51.147827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:35043 (local address 127.3.61.131:37977)
0811 19:12:51.148134 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:51.148158 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:51.148320 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:51.149629 (+ 1309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:51.149635 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:51.149747 (+ 112us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:12:51.150013 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:51.150028 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:51.151280 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:51.151286 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:51.152786 (+ 1500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:51.152802 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:51.154068 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:51.154074 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:51.154259 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:51.154288 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 19:12:51.154378 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 19:12:51.154540 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"mutex_wait_us":39}
I20260811 19:12:51.155383 22276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:51.148581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:33005 (local address 127.3.61.130:42025)
0811 19:12:51.148834 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:51.148841 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:51.149223 (+ 382us) server_negotiation.cc:407] Connection header received
0811 19:12:51.149432 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:51.149439 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:51.149511 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:51.149598 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:12:51.150548 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:51.151331 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:51.153290 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:51.154267 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:51.154461 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:51.154547 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:51.154851 (+ 304us) server_negotiation.cc:299] Negotiation successful
0811 19:12:51.154998 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 19:12:51.155545 22290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:51.148205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.130:42025 (local address 127.3.61.131:33005)
0811 19:12:51.149105 (+ 900us) negotiation.cc:107] Waiting for socket to connect
0811 19:12:51.149120 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 19:12:51.149288 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:12:51.149633 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:12:51.149637 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:12:51.149691 (+ 54us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:12:51.150347 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:51.150361 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:51.151518 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:51.151524 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:51.153093 (+ 1569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:12:51.153108 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:12:51.154502 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:12:51.154509 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:12:51.154680 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:51.154704 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 19:12:51.154804 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 19:12:51.154954 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":791,"mutex_wait_us":21,"thread_start_us":385,"threads_started":1}
I20260811 19:12:51.156836 21769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bfdf998f23d492fb8e7c559fede96bb" candidate_uuid: "167fbdde16bc424a8961aab5463ba4df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73acf2a5e2e44b178dbcdceff4438c71" is_pre_election: true
I20260811 19:12:51.157827 21769 raft_consensus.cc:2454] T 4bfdf998f23d492fb8e7c559fede96bb P 73acf2a5e2e44b178dbcdceff4438c71 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 167fbdde16bc424a8961aab5463ba4df in term 0.
I20260811 19:12:51.157758 21974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bfdf998f23d492fb8e7c559fede96bb" candidate_uuid: "167fbdde16bc424a8961aab5463ba4df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "975ccb3c32144fdb8c932958cce7c9e0" is_pre_election: true
I20260811 19:12:51.158389 21974 raft_consensus.cc:2454] T 4bfdf998f23d492fb8e7c559fede96bb P 975ccb3c32144fdb8c932958cce7c9e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 167fbdde16bc424a8961aab5463ba4df in term 0.
I20260811 19:12:51.159061 22089 leader_election.cc:302] T 4bfdf998f23d492fb8e7c559fede96bb P 167fbdde16bc424a8961aab5463ba4df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 167fbdde16bc424a8961aab5463ba4df, 73acf2a5e2e44b178dbcdceff4438c71; no voters:
I20260811 19:12:51.159724 22289 raft_consensus.cc:2781] T 4bfdf998f23d492fb8e7c559fede96bb P 167fbdde16bc424a8961aab5463ba4df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:12:51.159926 22289 raft_consensus.cc:482] T 4bfdf998f23d492fb8e7c559fede96bb P 167fbdde16bc424a8961aab5463ba4df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:12:51.160073 22289 raft_consensus.cc:3037] T 4bfdf998f23d492fb8e7c559fede96bb P 167fbdde16bc424a8961aab5463ba4df [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:12:51.164151 22289 raft_consensus.cc:504] T 4bfdf998f23d492fb8e7c559fede96bb P 167fbdde16bc424a8961aab5463ba4df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73acf2a5e2e44b178dbcdceff4438c71" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 35043 } } peers { permanent_uuid: "975ccb3c32144fdb8c932958cce7c9e0" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 42025 } } peers { permanent_uuid: "167fbdde16bc424a8961aab5463ba4df" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 43187 } }
I20260811 19:12:51.165006 22289 leader_election.cc:288] T 4bfdf998f23d492fb8e7c559fede96bb P 167fbdde16bc424a8961aab5463ba4df [CANDIDATE]: Term 1 election: Requested vote from peers 73acf2a5e2e44b178dbcdceff4438c71 (127.3.61.129:35043), 975ccb3c32144fdb8c932958cce7c9e0 (127.3.61.130:42025)
I20260811 19:12:51.165705 21769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bfdf998f23d492fb8e7c559fede96bb" candidate_uuid: "167fbdde16bc424a8961aab5463ba4df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73acf2a5e2e44b178dbcdceff4438c71"
I20260811 19:12:51.165766 21974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bfdf998f23d492fb8e7c559fede96bb" candidate_uuid: "167fbdde16bc424a8961aab5463ba4df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "975ccb3c32144fdb8c932958cce7c9e0"
I20260811 19:12:51.165967 21769 raft_consensus.cc:3037] T 4bfdf998f23d492fb8e7c559fede96bb P 73acf2a5e2e44b178dbcdceff4438c71 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:12:51.165992 21974 raft_consensus.cc:3037] T 4bfdf998f23d492fb8e7c559fede96bb P 975ccb3c32144fdb8c932958cce7c9e0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:12:51.168303 21769 raft_consensus.cc:2454] T 4bfdf998f23d492fb8e7c559fede96bb P 73acf2a5e2e44b178dbcdceff4438c71 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 167fbdde16bc424a8961aab5463ba4df in term 1.
I20260811 19:12:51.168414 21974 raft_consensus.cc:2454] T 4bfdf998f23d492fb8e7c559fede96bb P 975ccb3c32144fdb8c932958cce7c9e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 167fbdde16bc424a8961aab5463ba4df in term 1.
I20260811 19:12:51.168826 22089 leader_election.cc:302] T 4bfdf998f23d492fb8e7c559fede96bb P 167fbdde16bc424a8961aab5463ba4df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 167fbdde16bc424a8961aab5463ba4df, 73acf2a5e2e44b178dbcdceff4438c71; no voters:
I20260811 19:12:51.169148 22289 raft_consensus.cc:2781] T 4bfdf998f23d492fb8e7c559fede96bb P 167fbdde16bc424a8961aab5463ba4df [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:12:51.169831 22289 raft_consensus.cc:686] T 4bfdf998f23d492fb8e7c559fede96bb P 167fbdde16bc424a8961aab5463ba4df [term 1 LEADER]: Becoming Leader. State: Replica: 167fbdde16bc424a8961aab5463ba4df, State: Running, Role: LEADER
I20260811 19:12:51.170639 22289 consensus_queue.cc:237] T 4bfdf998f23d492fb8e7c559fede96bb P 167fbdde16bc424a8961aab5463ba4df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73acf2a5e2e44b178dbcdceff4438c71" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 35043 } } peers { permanent_uuid: "975ccb3c32144fdb8c932958cce7c9e0" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 42025 } } peers { permanent_uuid: "167fbdde16bc424a8961aab5463ba4df" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 43187 } }
I20260811 19:12:51.177203 21522 catalog_manager.cc:5495] T 4bfdf998f23d492fb8e7c559fede96bb P 167fbdde16bc424a8961aab5463ba4df reported cstate change: term changed from 0 to 1, leader changed from <none> to 167fbdde16bc424a8961aab5463ba4df (127.3.61.131). New cstate: current_term: 1 leader_uuid: "167fbdde16bc424a8961aab5463ba4df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73acf2a5e2e44b178dbcdceff4438c71" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 35043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "975ccb3c32144fdb8c932958cce7c9e0" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 42025 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "167fbdde16bc424a8961aab5463ba4df" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 43187 } health_report { overall_health: HEALTHY } } }
W20260811 19:12:51.212239 22254 tablet.cc:2369] T 4bfdf998f23d492fb8e7c559fede96bb P 167fbdde16bc424a8961aab5463ba4df: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:12:51.220585 22275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:51.213266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38280 (local address 127.3.61.131:43187)
0811 19:12:51.213458 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:51.213463 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:51.213743 (+ 280us) server_negotiation.cc:407] Connection header received
0811 19:12:51.213969 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:51.213974 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:51.214123 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:51.214391 (+ 268us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:12:51.214885 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:51.215891 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:51.217526 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:51.218087 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:51.218214 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:51.218857 (+ 643us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:12:51.219736 (+ 879us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:12:51.219934 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:51.220193 (+ 259us) server_negotiation.cc:299] Negotiation successful
0811 19:12:51.220307 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 19:12:51.231889 21974 raft_consensus.cc:1261] T 4bfdf998f23d492fb8e7c559fede96bb P 975ccb3c32144fdb8c932958cce7c9e0 [term 1 FOLLOWER]: Refusing update from remote peer 167fbdde16bc424a8961aab5463ba4df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:12:51.231998 21769 raft_consensus.cc:1261] T 4bfdf998f23d492fb8e7c559fede96bb P 73acf2a5e2e44b178dbcdceff4438c71 [term 1 FOLLOWER]: Refusing update from remote peer 167fbdde16bc424a8961aab5463ba4df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:12:51.233323 22292 consensus_queue.cc:1035] T 4bfdf998f23d492fb8e7c559fede96bb P 167fbdde16bc424a8961aab5463ba4df [LEADER]: Connected to new peer: Peer: permanent_uuid: "73acf2a5e2e44b178dbcdceff4438c71" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 35043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:12:51.233834 22289 consensus_queue.cc:1035] T 4bfdf998f23d492fb8e7c559fede96bb P 167fbdde16bc424a8961aab5463ba4df [LEADER]: Connected to new peer: Peer: permanent_uuid: "975ccb3c32144fdb8c932958cce7c9e0" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 42025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:12:51.247746 22298 mvcc.cc:205] Tried to move back new op lower bound from 7317403939751493632 to 7317403939520520192. Current Snapshot: MvccSnapshot[applied={T|T < 7317403939751493632}]
I20260811 19:12:51.250155 22300 mvcc.cc:205] Tried to move back new op lower bound from 7317403939751493632 to 7317403939520520192. Current Snapshot: MvccSnapshot[applied={T|T < 7317403939751493632}]
I20260811 19:12:51.264567 21522 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48316:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:12:51.264940 21522 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48316:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:12:51.269896 21522 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b2c1e2dfb77a49c6a8ff944061042be9: Sending DeleteTablet for 3 replicas of tablet 4bfdf998f23d492fb8e7c559fede96bb
I20260811 19:12:51.271963 22151 tablet_service.cc:1514] Processing DeleteTablet for tablet 4bfdf998f23d492fb8e7c559fede96bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:12:51 UTC) from {username='kudu', principal='kudu/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:38266
I20260811 19:12:51.272265 21734 tablet_service.cc:1514] Processing DeleteTablet for tablet 4bfdf998f23d492fb8e7c559fede96bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:12:51 UTC) from {username='kudu', principal='kudu/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:39174
I20260811 19:12:51.272373 21944 tablet_service.cc:1514] Processing DeleteTablet for tablet 4bfdf998f23d492fb8e7c559fede96bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:12:51 UTC) from {username='kudu', principal='kudu/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:49540
I20260811 19:12:51.278383 22315 tablet_replica.cc:330] T 4bfdf998f23d492fb8e7c559fede96bb P 975ccb3c32144fdb8c932958cce7c9e0: stopping tablet replica
I20260811 19:12:51.278479 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 21630
I20260811 19:12:51.282357 22315 raft_consensus.cc:2229] T 4bfdf998f23d492fb8e7c559fede96bb P 975ccb3c32144fdb8c932958cce7c9e0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:12:51.283090 22315 raft_consensus.cc:2258] T 4bfdf998f23d492fb8e7c559fede96bb P 975ccb3c32144fdb8c932958cce7c9e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:12:51.284521 22314 tablet_replica.cc:330] T 4bfdf998f23d492fb8e7c559fede96bb P 73acf2a5e2e44b178dbcdceff4438c71: stopping tablet replica
I20260811 19:12:51.285055 22314 raft_consensus.cc:2229] T 4bfdf998f23d492fb8e7c559fede96bb P 73acf2a5e2e44b178dbcdceff4438c71 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:12:51.285594 22315 ts_tablet_manager.cc:1905] T 4bfdf998f23d492fb8e7c559fede96bb P 975ccb3c32144fdb8c932958cce7c9e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:12:51.285681 22314 raft_consensus.cc:2258] T 4bfdf998f23d492fb8e7c559fede96bb P 73acf2a5e2e44b178dbcdceff4438c71 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:12:51.287958 22314 ts_tablet_manager.cc:1905] T 4bfdf998f23d492fb8e7c559fede96bb P 73acf2a5e2e44b178dbcdceff4438c71: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:12:51.294034 22315 ts_tablet_manager.cc:1918] T 4bfdf998f23d492fb8e7c559fede96bb P 975ccb3c32144fdb8c932958cce7c9e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:12:51.294250 22315 log.cc:1196] T 4bfdf998f23d492fb8e7c559fede96bb P 975ccb3c32144fdb8c932958cce7c9e0: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/ts-1/wal/wals/4bfdf998f23d492fb8e7c559fede96bb
I20260811 19:12:51.294876 22315 ts_tablet_manager.cc:1939] T 4bfdf998f23d492fb8e7c559fede96bb P 975ccb3c32144fdb8c932958cce7c9e0: Deleting consensus metadata
I20260811 19:12:51.295807 22314 ts_tablet_manager.cc:1918] T 4bfdf998f23d492fb8e7c559fede96bb P 73acf2a5e2e44b178dbcdceff4438c71: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:12:51.295993 22314 log.cc:1196] T 4bfdf998f23d492fb8e7c559fede96bb P 73acf2a5e2e44b178dbcdceff4438c71: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/ts-0/wal/wals/4bfdf998f23d492fb8e7c559fede96bb
I20260811 19:12:51.296577 22314 ts_tablet_manager.cc:1939] T 4bfdf998f23d492fb8e7c559fede96bb P 73acf2a5e2e44b178dbcdceff4438c71: Deleting consensus metadata
I20260811 19:12:51.297339 21501 catalog_manager.cc:4841] TS 975ccb3c32144fdb8c932958cce7c9e0 (127.3.61.130:42025): tablet 4bfdf998f23d492fb8e7c559fede96bb (table test-table [id=01b850beacec407f926097378a995207]) successfully deleted
I20260811 19:12:51.297614 22313 tablet_replica.cc:330] T 4bfdf998f23d492fb8e7c559fede96bb P 167fbdde16bc424a8961aab5463ba4df: stopping tablet replica
I20260811 19:12:51.298180 22313 raft_consensus.cc:2229] T 4bfdf998f23d492fb8e7c559fede96bb P 167fbdde16bc424a8961aab5463ba4df [term 1 LEADER]: Raft consensus shutting down.
I20260811 19:12:51.299031 22313 raft_consensus.cc:2258] T 4bfdf998f23d492fb8e7c559fede96bb P 167fbdde16bc424a8961aab5463ba4df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:12:51.299767 21501 catalog_manager.cc:4841] TS 73acf2a5e2e44b178dbcdceff4438c71 (127.3.61.129:35043): tablet 4bfdf998f23d492fb8e7c559fede96bb (table test-table [id=01b850beacec407f926097378a995207]) successfully deleted
Aug 11 19:12:51 dist-test-slave-t7g5 krb5kdc[21449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.129@KRBTEST.COM
I20260811 19:12:51.302837 22313 ts_tablet_manager.cc:1905] T 4bfdf998f23d492fb8e7c559fede96bb P 167fbdde16bc424a8961aab5463ba4df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:12:51.313719 22313 ts_tablet_manager.cc:1918] T 4bfdf998f23d492fb8e7c559fede96bb P 167fbdde16bc424a8961aab5463ba4df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:12:51.313961 22313 log.cc:1196] T 4bfdf998f23d492fb8e7c559fede96bb P 167fbdde16bc424a8961aab5463ba4df: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786475454804547-3318-0/minicluster-data/ts-2/wal/wals/4bfdf998f23d492fb8e7c559fede96bb
I20260811 19:12:51.314252 22271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:51.280126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39180 (local address 127.3.61.129:35043)
0811 19:12:51.280345 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:51.280352 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:51.282545 (+ 2193us) server_negotiation.cc:407] Connection header received
0811 19:12:51.283752 (+ 1207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:51.283760 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:51.283896 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:51.283983 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:51.286185 (+ 2202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:51.287302 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:51.289031 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:51.289716 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:51.289837 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:51.303203 (+ 13366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:51.303233 (+ 30us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:51.303256 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:51.303326 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:51.308934 (+ 5608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:51.310811 (+ 1877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:51.310825 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:51.310828 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:51.311074 (+ 246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:51.311628 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:51.311633 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:51.311636 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:51.312198 (+ 562us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:51.312534 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:51.313237 (+ 703us) server_negotiation.cc:299] Negotiation successful
0811 19:12:51.313547 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 19:12:51.314740 22313 ts_tablet_manager.cc:1939] T 4bfdf998f23d492fb8e7c559fede96bb P 167fbdde16bc424a8961aab5463ba4df: Deleting consensus metadata
I20260811 19:12:51.316509 21828 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:12:51.316992 21495 catalog_manager.cc:4841] TS 167fbdde16bc424a8961aab5463ba4df (127.3.61.131:43187): tablet 4bfdf998f23d492fb8e7c559fede96bb (table test-table [id=01b850beacec407f926097378a995207]) successfully deleted
I20260811 19:12:51.471801 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 21630
I20260811 19:12:51.498517 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 21842
Aug 11 19:12:51 dist-test-slave-t7g5 krb5kdc[21449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.130@KRBTEST.COM
I20260811 19:12:51.517587 22276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:51.499413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49542 (local address 127.3.61.130:42025)
0811 19:12:51.499612 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:51.499619 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:51.499639 (+ 20us) server_negotiation.cc:407] Connection header received
0811 19:12:51.499794 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:51.499799 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:51.499878 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:51.499967 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:51.501232 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:51.502084 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:51.503349 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:51.503811 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:51.503926 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:51.509995 (+ 6069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:51.510036 (+ 41us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:51.510066 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:51.510129 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:51.513625 (+ 3496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:51.514996 (+ 1371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:51.515009 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:51.515012 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:51.515230 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:51.515739 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:51.515743 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:51.515745 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:51.516219 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:51.516519 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:51.516902 (+ 383us) server_negotiation.cc:299] Negotiation successful
0811 19:12:51.517146 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 19:12:51.518350 22038 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:12:51.619493 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 21842
I20260811 19:12:51.652598 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 22054
Aug 11 19:12:51 dist-test-slave-t7g5 krb5kdc[21449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.131@KRBTEST.COM
I20260811 19:12:51.675637 22275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:12:51.653691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38288 (local address 127.3.61.131:43187)
0811 19:12:51.653888 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 19:12:51.653894 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:12:51.653931 (+ 37us) server_negotiation.cc:407] Connection header received
0811 19:12:51.654152 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:12:51.654156 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:12:51.654223 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:12:51.654309 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:12:51.655708 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:51.656560 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:51.658619 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:12:51.659099 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:12:51.659214 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:12:51.665617 (+ 6403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:12:51.665649 (+ 32us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:12:51.665779 (+ 130us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:12:51.665867 (+ 88us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:12:51.671170 (+ 5303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:51.672371 (+ 1201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:51.672382 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:51.672386 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:51.672629 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:12:51.673344 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:12:51.673350 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:12:51.673354 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:12:51.674088 (+ 734us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:12:51.674499 (+ 411us) server_negotiation.cc:1035] Waiting for connection context
0811 19:12:51.674896 (+ 397us) server_negotiation.cc:299] Negotiation successful
0811 19:12:51.675194 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 19:12:51.676795 22245 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:12:51.814271 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 22054
I20260811 19:12:51.845034 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 21465
I20260811 19:12:51.846611 21582 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:12:51.964211 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 21465
2026-08-11T19:12:51Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (5397 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:12:52 dist-test-slave-t7g5 krb5kdc[22326](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:12:52 dist-test-slave-t7g5 krb5kdc[22326](info): set up 2 sockets
Aug 11 19:12:52 dist-test-slave-t7g5 krb5kdc[22326](info): commencing operation
krb5kdc: starting...
W20260811 19:12:53.303922 3318 mini_kdc.cc:121] Time spent starting KDC: real 1.285s user 0.003s sys 0.057s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Aug 11 19:12:53 dist-test-slave-t7g5 krb5kdc[22326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475573, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:12:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:12:53Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.3.61.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.3.61.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.3.61.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/krb5kdc/rangeradmin_127.3.61.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.3.61.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/krb5kdc/rangeradmin_127.3.61.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.3.61.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.3.61.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.3.61.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/krb5kdc/rangerlookup_127.3.61.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.3.61.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/krb5kdc/rangerlookup_127.3.61.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.3.61.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.3.61.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.3.61.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/krb5kdc/HTTP_127.3.61.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.3.61.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/krb5kdc/HTTP_127.3.61.148@KRBTEST.COM.keytab.
I20260811 19:12:53.841130 3318 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskwEqXVd/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/postgres -l logfile start
2026-08-11 19:12:56.746 UTC [22376] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 19:12:56.746 UTC [22376] LOG: listening on IPv4 address "127.3.61.148", port 52077
2026-08-11 19:12:56.750 UTC [22376] LOG: listening on Unix socket "/tmp/.s.PGSQL.52077"
2026-08-11 19:12:56.775 UTC [22379] LOG: database system was shut down at 2026-08-11 19:12:55 UTC
2026-08-11 19:12:56.779 UTC [22376] LOG: database system is ready to accept connections
I20260811 19:12:57.977481 3318 mini_postgres.cc:96] Postgres bound to 52077
2026-08-11 19:12:58.007 UTC [22395] FATAL: database "slave" does not exist
127.3.61.148:52077 - accepting connections
I20260811 19:12:58.008563 3318 mini_ranger.cc:161] Starting Ranger...
I20260811 19:12:58.050806 3318 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 19:12:58.660980 3318 mini_ranger.cc:87] Created ranger Postgres database
I20260811 19:12:58.661113 3318 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ranger-admin
2026-08-11 19:12:59,085 [I] DB FLAVOR :POSTGRES
2026-08-11 19:12:59,086 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 19:12:59,086 [I] Checking connection..
2026-08-11 19:12:59,086 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwEqXVd/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwEqXVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.61.148:52077/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 19:12:59,409 [I] Checking connection passed.
2026-08-11 19:12:59,409 [I] --------- Verifying version history table ---------
2026-08-11 19:12:59,410 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwEqXVd/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwEqXVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.61.148:52077/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 19:12:59,751 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 19:12:59,751 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwEqXVd/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwEqXVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.61.148:52077/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 19:13:00,041 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 19:13:00,042 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 19:13:00,042 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwEqXVd/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwEqXVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.61.148:52077/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskwEqXVd/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 19:13:00.334 UTC [22487] WARNING: there is no transaction in progress
2026-08-11 19:13:00,341 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 19:13:00,341 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwEqXVd/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwEqXVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.61.148:52077/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 19:13:00,659 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 19:13:00,659 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 19:13:00,660 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwEqXVd/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwEqXVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.61.148:52077/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 19:13:00,944 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwEqXVd/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwEqXVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.61.148:52077/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 19:13:01,232 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwEqXVd/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwEqXVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.61.148:52077/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-t7g5.c.gcp-upstream.internal','N') ;"
2026-08-11 19:13:01,503 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwEqXVd/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwEqXVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.61.148:52077/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 19:13:01,798 [I] Table x_portal_user does not exist in database ranger
2026-08-11 19:13:01,799 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwEqXVd/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwEqXVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.61.148:52077/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 19:13:02,091 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 19:13:02,091 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwEqXVd/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwEqXVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.61.148:52077/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 19:13:02,377 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwEqXVd/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwEqXVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.61.148:52077/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 19:13:02,674 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 19:13:02,674 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwEqXVd/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwEqXVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.61.148:52077/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskwEqXVd/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 19:13:03.898 UTC [22660] WARNING: there is no transaction in progress
2026-08-11 19:13:03.920 UTC [22660] WARNING: there is no transaction in progress
2026-08-11 19:13:03.941 UTC [22660] WARNING: there is no transaction in progress
2026-08-11 19:13:03.962 UTC [22660] WARNING: there is no transaction in progress
2026-08-11 19:13:03.984 UTC [22660] WARNING: there is no transaction in progress
2026-08-11 19:13:04.005 UTC [22660] WARNING: there is no transaction in progress
2026-08-11 19:13:04.076 UTC [22660] WARNING: there is no transaction in progress
2026-08-11 19:13:04.085 UTC [22660] WARNING: there is no transaction in progress
2026-08-11 19:13:04.104 UTC [22660] WARNING: there is no transaction in progress
2026-08-11 19:13:04.117 UTC [22660] WARNING: there is no transaction in progress
2026-08-11 19:13:04.131 UTC [22660] WARNING: there is no transaction in progress
2026-08-11 19:13:04.146 UTC [22660] WARNING: there is no transaction in progress
2026-08-11 19:13:04.160 UTC [22660] WARNING: there is no transaction in progress
2026-08-11 19:13:04.169 UTC [22660] WARNING: there is no transaction in progress
2026-08-11 19:13:04.178 UTC [22660] WARNING: there is no transaction in progress
2026-08-11 19:13:04.187 UTC [22660] WARNING: there is no transaction in progress
2026-08-11 19:13:05,103 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 19:13:05,104 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwEqXVd/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwEqXVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.61.148:52077/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 19:13:05,403 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 19:13:05,404 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwEqXVd/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwEqXVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.61.148:52077/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 19:13:05,702 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 19:13:05,702 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwEqXVd/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwEqXVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.61.148:52077/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 19:13:05,998 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 19:13:05,999 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwEqXVd/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwEqXVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.61.148:52077/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 19:13:06,299 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwEqXVd/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwEqXVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.61.148:52077/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 19:13:06,598 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwEqXVd/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwEqXVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.61.148:52077/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-t7g5.c.gcp-upstream.internal';"
2026-08-11 19:13:06,910 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 19:13:06,910 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwEqXVd/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwEqXVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.61.148:52077/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 19:13:07,239 [I] DB_PATCHES have already been applied
I20260811 19:13:07.246966 3318 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ranger-admin:/tmp/dist-test-taskwEqXVd/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskwEqXVd/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskwEqXVd/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwEqXVd/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 19:13:07.247073 3318 mini_ranger.cc:193] Using host: 127.3.61.148
I20260811 19:13:07.281181 3318 mini_ranger.cc:227] Ranger admin URL: http://127.3.61.148:59331
Aug 11, 2026 7:13:07 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 7:13:07 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 7:13:07 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 7:13:07 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 7:13:07 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 7:13:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 7:13:08 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 19:13:08.714309 3318 mini_ranger.cc:160] Time spent starting Ranger: real 10.706s user 0.005s sys 0.096s
Aug 11, 2026 7:13:33 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 7:13:33 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 11, 2026 7:13:33 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 7:13:33 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 7:13:33 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 7:13:33 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 7:13:33 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 7:13:33 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 7:13:33 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 7:13:33 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 7:13:33 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 7:13:33 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 7:13:33 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 7:13:33 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 7:13:33 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 7:13:33 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 7:13:33 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 7:13:34 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 19:13:35.578765 3318 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.3.61.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.3.61.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.3.61.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/krb5kdc/rangerkms_127.3.61.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.3.61.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/krb5kdc/rangerkms_127.3.61.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.3.61.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.3.61.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.3.61.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/krb5kdc/HTTP_127.3.61.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.3.61.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/krb5kdc/HTTP_127.3.61.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 19:13:35 dist-test-slave-t7g5 krb5kdc[22326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475615, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 19:13:35.848553 3318 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 19:13:35.882654 3318 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 19:13:36.674458 3318 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 19:13:36.674642 3318 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ranger-kms
I20260811 19:13:36.674780 3318 mini_ranger_kms.cc:227] Using postgres at 127.3.61.148:52077
2026-08-11 19:13:37,304 [I] DB FLAVOR :POSTGRES
2026-08-11 19:13:37,304 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 19:13:37,304 [I] Checking connection
2026-08-11 19:13:37,304 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwEqXVd/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.61.148:52077/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 19:13:37,593 [I] connection success
2026-08-11 19:13:37,593 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 19:13:37,593 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 19:13:37,594 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwEqXVd/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.61.148:52077/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 19:13:37,891 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 19:13:37,891 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 19:13:37,891 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 19:13:37,891 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwEqXVd/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.61.148:52077/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 19:13:38,204 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 19:13:38.833108 3318 mini_ranger_kms.cc:310] Created kms service
I20260811 19:13:39.075235 3318 mini_ranger_kms.cc:326] Created kudu user
I20260811 19:13:39.130679 3318 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 19:13:39 dist-test-slave-t7g5 krb5kdc[22326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475615, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.3.61.148@KRBTEST.COM
I20260811 19:13:39.620725 3318 mini_ranger_kms.cc:384] Added ranger policy
I20260811 19:13:39.624081 3318 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ranger-kms:/tmp/dist-test-taskwEqXVd/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwEqXVd/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskwEqXVd/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 19:13:39.624202 3318 mini_ranger_kms.cc:240] Using host: 127.3.61.148
I20260811 19:13:39.663290 3318 mini_ranger_kms.cc:276] Ranger KMS PID: 22946
I20260811 19:13:39.663666 3318 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.3.61.148:36437
Aug 11, 2026 7:13:40 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 7:13:40 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 7:13:41 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 7:13:41 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 7:13:41 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 7:13:41 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 19:13:42 dist-test-slave-t7g5 krb5kdc[22326](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786475622, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.3.61.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:13:42.407269 3318 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.559s user 0.184s sys 0.507s
I20260811 19:13:42.407637 3318 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 19:13:42.407879 3318 mini_ranger_kms.cc:401] 127.3.61.148:36437/kms/v1/keys
Aug 11 19:13:42 dist-test-slave-t7g5 krb5kdc[22326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475622, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.3.61.148@KRBTEST.COM for HTTP/127.3.61.148@KRBTEST.COM
Aug 11 19:13:43 dist-test-slave-t7g5 krb5kdc[22326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475622, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.3.61.148@KRBTEST.COM for HTTP/127.3.61.148@KRBTEST.COM
Aug 11 19:13:45 dist-test-slave-t7g5 krb5kdc[22326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475625, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.3.61.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab.
Entry for principal kudu/127.3.61.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab.
I20260811 19:13:45.790664 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:36747
--webserver_interface=127.3.61.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:36393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.61.190:36747
--ranger_config_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.61.148:36437/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:13:46.322098 23020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:13:46.322501 23020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:13:46.322718 23020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:13:46.333104 23020 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:13:46.333379 23020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:13:46.333554 23020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:13:46.333747 23020 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:13:46.333948 23020 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:13:46.349431 23020 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:36393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/master-0/wal
--ranger_kms_url=127.3.61.148:36437/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.3.61.190:36747
--ranger_config_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:36747
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.61.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:13:46.351116 23020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:13:46.352838 23020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:13:46.369004 23030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:13:46.368969 23032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:13:46.368829 23028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:13:46.370386 23020 server_base.cc:1034] running on GCE node
I20260811 19:13:46.371733 23020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:13:46.374133 23020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:13:46.375523 23020 hybrid_clock.cc:648] HybridClock initialized: now 1786475626375415 us; error 48 us; skew 500 ppm
I20260811 19:13:46.376233 23020 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:13:46 dist-test-slave-t7g5 krb5kdc[22326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475626, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:13:46.384519 23020 init.cc:377] Logged in from keytab as kudu/127.3.61.190@KRBTEST.COM (short username kudu)
I20260811 19:13:46.386543 23020 webserver.cc:460] Webserver started at http://127.3.61.190:41825/ using document root <none> and password file <none>
I20260811 19:13:46.387492 23020 fs_manager.cc:359] Metadata directory not provided
I20260811 19:13:46.387706 23020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:13:46.388083 23020 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 19:13:46 dist-test-slave-t7g5 krb5kdc[22326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475626, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.190@KRBTEST.COM for HTTP/127.3.61.148@KRBTEST.COM
I20260811 19:13:46.463403 23020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/master-0/data/instance:
uuid: "e3d4b43916084f5bb2d4bf0041873fde"
format_stamp: "Formatted at 2026-08-11 19:13:46 on dist-test-slave-t7g5"
server_key: "8eff76a5448c50c419ea8b6dbe844c09"
server_key_iv: "ec289d3a28affeb6e25c89bcd5dc4778"
server_key_version: "kuduclusterkey@0"
I20260811 19:13:46.464419 23020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/master-0/wal/instance:
uuid: "e3d4b43916084f5bb2d4bf0041873fde"
format_stamp: "Formatted at 2026-08-11 19:13:46 on dist-test-slave-t7g5"
server_key: "8eff76a5448c50c419ea8b6dbe844c09"
server_key_iv: "ec289d3a28affeb6e25c89bcd5dc4778"
server_key_version: "kuduclusterkey@0"
I20260811 19:13:46.472026 23020 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.004s
I20260811 19:13:46.492266 23043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:13:46.495225 23020 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.005s sys 0.000s
I20260811 19:13:46.495561 23020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/master-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/master-0/wal
uuid: "e3d4b43916084f5bb2d4bf0041873fde"
format_stamp: "Formatted at 2026-08-11 19:13:46 on dist-test-slave-t7g5"
server_key: "8eff76a5448c50c419ea8b6dbe844c09"
server_key_iv: "ec289d3a28affeb6e25c89bcd5dc4778"
server_key_version: "kuduclusterkey@0"
I20260811 19:13:46.495929 23020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:13:46.515657 23020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:13:46.521003 23020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:13:46.521400 23020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:13:46.554979 23020 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.190:36747
I20260811 19:13:46.555012 23131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.190:36747 every 8 connection(s)
I20260811 19:13:46.556777 23020 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
I20260811 19:13:46.561508 23133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:13:46.566157 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 23020
I20260811 19:13:46.566368 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/master-0/wal/instance
I20260811 19:13:46.567193 3318 external_mini_cluster.cc:1428] Setting key a4d55c8f6ea67aee33c0a14794ae6623
I20260811 19:13:46.576713 23133 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3d4b43916084f5bb2d4bf0041873fde: Bootstrap starting.
I20260811 19:13:46.580554 23133 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e3d4b43916084f5bb2d4bf0041873fde: Neither blocks nor log segments found. Creating new log.
I20260811 19:13:46.581971 23133 log.cc:824] T 00000000000000000000000000000000 P e3d4b43916084f5bb2d4bf0041873fde: Log is configured to *not* fsync() on all Append() calls
Aug 11 19:13:46 dist-test-slave-t7g5 krb5kdc[22326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475625, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:13:46.586172 23133 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3d4b43916084f5bb2d4bf0041873fde: No bootstrap required, opened a new log
I20260811 19:13:46.597244 23133 raft_consensus.cc:348] T 00000000000000000000000000000000 P e3d4b43916084f5bb2d4bf0041873fde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3d4b43916084f5bb2d4bf0041873fde" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 36747 } }
I20260811 19:13:46.597815 23133 raft_consensus.cc:374] T 00000000000000000000000000000000 P e3d4b43916084f5bb2d4bf0041873fde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:13:46.597975 23133 raft_consensus.cc:729] T 00000000000000000000000000000000 P e3d4b43916084f5bb2d4bf0041873fde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3d4b43916084f5bb2d4bf0041873fde, State: Initialized, Role: FOLLOWER
I20260811 19:13:46.598440 23137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:13:46.570198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37022 (local address 127.3.61.190:36747)
0811 19:13:46.571493 (+ 1295us) server_negotiation.cc:206] Beginning negotiation
0811 19:13:46.571507 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 19:13:46.571622 (+ 115us) server_negotiation.cc:407] Connection header received
0811 19:13:46.574907 (+ 3285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:13:46.574944 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:13:46.575485 (+ 541us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:13:46.575933 (+ 448us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:13:46.577459 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:13:46.579067 (+ 1608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:13:46.580339 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:13:46.581208 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:13:46.581390 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:13:46.586038 (+ 4648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:13:46.586049 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:13:46.586067 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:13:46.586107 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:13:46.590979 (+ 4872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:13:46.593375 (+ 2396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:13:46.593384 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:13:46.593393 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:13:46.593723 (+ 330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:13:46.594219 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:13:46.594223 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:13:46.594225 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:13:46.595137 (+ 912us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:13:46.595721 (+ 584us) server_negotiation.cc:1035] Waiting for connection context
0811 19:13:46.596513 (+ 792us) server_negotiation.cc:299] Negotiation successful
0811 19:13:46.597039 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1002,"thread_start_us":718,"threads_started":1}
I20260811 19:13:46.598912 23133 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3d4b43916084f5bb2d4bf0041873fde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3d4b43916084f5bb2d4bf0041873fde" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 36747 } }
I20260811 19:13:46.599318 23133 raft_consensus.cc:388] T 00000000000000000000000000000000 P e3d4b43916084f5bb2d4bf0041873fde [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:13:46.599545 23133 raft_consensus.cc:482] T 00000000000000000000000000000000 P e3d4b43916084f5bb2d4bf0041873fde [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:13:46.599797 23133 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e3d4b43916084f5bb2d4bf0041873fde [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:13:46.603863 23133 raft_consensus.cc:504] T 00000000000000000000000000000000 P e3d4b43916084f5bb2d4bf0041873fde [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3d4b43916084f5bb2d4bf0041873fde" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 36747 } }
I20260811 19:13:46.604559 23133 leader_election.cc:302] T 00000000000000000000000000000000 P e3d4b43916084f5bb2d4bf0041873fde [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e3d4b43916084f5bb2d4bf0041873fde; no voters:
I20260811 19:13:46.605958 23133 leader_election.cc:288] T 00000000000000000000000000000000 P e3d4b43916084f5bb2d4bf0041873fde [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:13:46.606207 23140 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e3d4b43916084f5bb2d4bf0041873fde [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:13:46.609726 23140 raft_consensus.cc:686] T 00000000000000000000000000000000 P e3d4b43916084f5bb2d4bf0041873fde [term 1 LEADER]: Becoming Leader. State: Replica: e3d4b43916084f5bb2d4bf0041873fde, State: Running, Role: LEADER
I20260811 19:13:46.610767 23133 sys_catalog.cc:564] T 00000000000000000000000000000000 P e3d4b43916084f5bb2d4bf0041873fde [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:13:46.610569 23140 consensus_queue.cc:237] T 00000000000000000000000000000000 P e3d4b43916084f5bb2d4bf0041873fde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3d4b43916084f5bb2d4bf0041873fde" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 36747 } }
I20260811 19:13:46.615694 23142 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3d4b43916084f5bb2d4bf0041873fde [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e3d4b43916084f5bb2d4bf0041873fde" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3d4b43916084f5bb2d4bf0041873fde" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 36747 } } }
I20260811 19:13:46.616456 23142 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3d4b43916084f5bb2d4bf0041873fde [sys.catalog]: This master's current role is: LEADER
I20260811 19:13:46.616760 23143 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3d4b43916084f5bb2d4bf0041873fde [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3d4b43916084f5bb2d4bf0041873fde. Latest consensus state: current_term: 1 leader_uuid: "e3d4b43916084f5bb2d4bf0041873fde" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3d4b43916084f5bb2d4bf0041873fde" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 36747 } } }
I20260811 19:13:46.617388 23143 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3d4b43916084f5bb2d4bf0041873fde [sys.catalog]: This master's current role is: LEADER
I20260811 19:13:46.620558 23153 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:13:46.628371 23153 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:13:46.629638 23133 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 19:13:46.651934 23153 catalog_manager.cc:1347] Generated new cluster ID: ac61a4a62c104e1eb0c7f730981b9425
I20260811 19:13:46.652146 23153 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:13:46.675879 23153 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:13:46.678332 23153 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:13:46.702947 23153 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e3d4b43916084f5bb2d4bf0041873fde: Generated new TSK 0
I20260811 19:13:46.704411 23153 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.61.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab.
Entry for principal kudu/127.3.61.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab.
I20260811 19:13:47.586822 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.61.148:36437/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.129:0
--local_ip_for_outbound_sockets=127.3.61.129
--webserver_interface=127.3.61.129
--webserver_port=0
--tserver_master_addrs=127.3.61.190:36747
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab
--principal=kudu/127.3.61.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:36393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 19:13:47 dist-test-slave-t7g5 krb5kdc[22326](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786475627, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 19:13:48.090467 23200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:13:48.090852 23200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:13:48.091055 23200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:13:48.101835 23200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:13:48.102042 23200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:13:48.102250 23200 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.129
I20260811 19:13:48.118613 23200 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:36393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.3.61.148:36437/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab
--principal=kudu/127.3.61.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.61.129
--webserver_port=0
--tserver_master_addrs=127.3.61.190:36747
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.129
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:13:48.120744 23200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:13:48.122660 23200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:13:48.144196 23216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:13:48.144265 23218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:13:48.145319 23215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:13:48.146543 23200 server_base.cc:1034] running on GCE node
I20260811 19:13:48.147297 23200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:13:48.148545 23200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:13:48.149818 23200 hybrid_clock.cc:648] HybridClock initialized: now 1786475628149736 us; error 47 us; skew 500 ppm
I20260811 19:13:48.150336 23200 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:13:48 dist-test-slave-t7g5 krb5kdc[22326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475628, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:13:48.155426 23200 init.cc:377] Logged in from keytab as kudu/127.3.61.129@KRBTEST.COM (short username kudu)
I20260811 19:13:48.158093 23200 webserver.cc:460] Webserver started at http://127.3.61.129:41503/ using document root <none> and password file <none>
I20260811 19:13:48.159384 23200 fs_manager.cc:359] Metadata directory not provided
I20260811 19:13:48.159618 23200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:13:48.160163 23200 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 19:13:48 dist-test-slave-t7g5 krb5kdc[22326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475628, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.129@KRBTEST.COM for HTTP/127.3.61.148@KRBTEST.COM
I20260811 19:13:48.184372 23200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ts-0/data/instance:
uuid: "087bd0e157ff44aebcc01a3617d8ddd5"
format_stamp: "Formatted at 2026-08-11 19:13:48 on dist-test-slave-t7g5"
server_key: "95c87987a53d14f4e59802bbb3db210d"
server_key_iv: "7ad9a4b3237917117a3f73dcfd866270"
server_key_version: "kuduclusterkey@0"
I20260811 19:13:48.185817 23200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ts-0/wal/instance:
uuid: "087bd0e157ff44aebcc01a3617d8ddd5"
format_stamp: "Formatted at 2026-08-11 19:13:48 on dist-test-slave-t7g5"
server_key: "95c87987a53d14f4e59802bbb3db210d"
server_key_iv: "7ad9a4b3237917117a3f73dcfd866270"
server_key_version: "kuduclusterkey@0"
I20260811 19:13:48.198801 23200 fs_manager.cc:691] Time spent creating directory manager: real 0.012s user 0.002s sys 0.009s
I20260811 19:13:48.245255 23226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:13:48.247646 23200 fs_manager.cc:725] Time spent opening block manager: real 0.029s user 0.015s sys 0.010s
I20260811 19:13:48.247972 23200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ts-0/wal
uuid: "087bd0e157ff44aebcc01a3617d8ddd5"
format_stamp: "Formatted at 2026-08-11 19:13:48 on dist-test-slave-t7g5"
server_key: "95c87987a53d14f4e59802bbb3db210d"
server_key_iv: "7ad9a4b3237917117a3f73dcfd866270"
server_key_version: "kuduclusterkey@0"
I20260811 19:13:48.248401 23200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:13:48.282933 23200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:13:48.290300 23200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:13:48.290867 23200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:13:48.292639 23200 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:13:48.295434 23200 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:13:48.295598 23200 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:13:48.295780 23200 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:13:48.295866 23200 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:13:48.411680 23200 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.129:43919
I20260811 19:13:48.411805 23399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.129:43919 every 8 connection(s)
I20260811 19:13:48.413863 23200 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
I20260811 19:13:48.414968 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 23200
I20260811 19:13:48.415184 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ts-0/wal/instance
I20260811 19:13:48.416077 3318 external_mini_cluster.cc:1428] Setting key bfe253ad8f173edecfb2289199f10b27
WARNING: no policy specified for kudu/127.3.61.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.130@KRBTEST.COM" created.
Aug 11 19:13:48 dist-test-slave-t7g5 krb5kdc[22326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475628, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.129@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:13:48.502882 23403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:13:48.446096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:50081 (local address 127.3.61.190:36747)
0811 19:13:48.454125 (+ 8029us) server_negotiation.cc:206] Beginning negotiation
0811 19:13:48.454133 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:13:48.463664 (+ 9531us) server_negotiation.cc:407] Connection header received
0811 19:13:48.476462 (+ 12798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:13:48.476471 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:13:48.476637 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:13:48.476757 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:13:48.480252 (+ 3495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:13:48.481603 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:13:48.484832 (+ 3229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:13:48.485718 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:13:48.485863 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:13:48.493170 (+ 7307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:13:48.493179 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:13:48.493184 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:13:48.493274 (+ 90us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:13:48.497934 (+ 4660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:13:48.499239 (+ 1305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:13:48.499246 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:13:48.499250 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:13:48.499518 (+ 268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:13:48.500190 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:13:48.500195 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:13:48.500198 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:13:48.500627 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:13:48.500752 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 19:13:48.501978 (+ 1226us) server_negotiation.cc:299] Negotiation successful
0811 19:13:48.502315 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7839,"thread_start_us":7689,"threads_started":1}
I20260811 19:13:48.503985 23404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:13:48.446359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:36747 (local address 127.3.61.129:50081)
0811 19:13:48.463415 (+ 17056us) negotiation.cc:107] Waiting for socket to connect
0811 19:13:48.463492 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0811 19:13:48.467774 (+ 4282us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:13:48.477609 (+ 9835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:13:48.477626 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:13:48.478647 (+ 1021us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:13:48.479915 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:13:48.479955 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:13:48.481987 (+ 2032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:13:48.481994 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:13:48.484564 (+ 2570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:13:48.484587 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:13:48.486396 (+ 1809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:13:48.486405 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:13:48.486867 (+ 462us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:13:48.487941 (+ 1074us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:13:48.487997 (+ 56us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:13:48.492789 (+ 4792us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:13:48.498174 (+ 5385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:13:48.498187 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:13:48.498208 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:13:48.499050 (+ 842us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:13:48.499705 (+ 655us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:13:48.499709 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:13:48.499712 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:13:48.500050 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:13:48.500863 (+ 813us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:13:48.500871 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:13:48.501381 (+ 510us) client_negotiation.cc:769] Sending connection context
0811 19:13:48.501759 (+ 378us) client_negotiation.cc:240] Negotiation successful
0811 19:13:48.502378 (+ 619us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16571,"thread_start_us":17173,"threads_started":1}
I20260811 19:13:48.506415 23400 heartbeater.cc:347] Connected to a master server at 127.3.61.190:36747
I20260811 19:13:48.506799 23400 heartbeater.cc:464] Registering TS with master...
I20260811 19:13:48.507807 23400 heartbeater.cc:511] Master 127.3.61.190:36747 requested a full tablet report, sending...
I20260811 19:13:48.511081 23073 ts_manager.cc:194] Registered new tserver with Master: 087bd0e157ff44aebcc01a3617d8ddd5 (127.3.61.129:43919)
I20260811 19:13:48.513629 23073 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.129@KRBTEST.COM'} at 127.3.61.129:50081
Aug 11 19:13:48 dist-test-slave-t7g5 krb5kdc[22326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475627, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.190@KRBTEST.COM for HTTP/127.3.61.148@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab.
Entry for principal kudu/127.3.61.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab.
W20260811 19:13:48.574182 3318 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.3.61.130: real 0.155s user 0.000s sys 0.034s
I20260811 19:13:48.575277 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.61.148:36437/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.130:0
--local_ip_for_outbound_sockets=127.3.61.130
--webserver_interface=127.3.61.130
--webserver_port=0
--tserver_master_addrs=127.3.61.190:36747
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab
--principal=kudu/127.3.61.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:36393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 19:13:48 dist-test-slave-t7g5 krb5kdc[22326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475627, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.190@KRBTEST.COM for HTTP/127.3.61.148@KRBTEST.COM
W20260811 19:13:49.311417 23406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:13:49.312026 23406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:13:49.312363 23406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:13:49.327906 23406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:13:49.328192 23406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:13:49.328461 23406 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.130
I20260811 19:13:49.344581 23406 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:36393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.3.61.148:36437/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab
--principal=kudu/127.3.61.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.61.130
--webserver_port=0
--tserver_master_addrs=127.3.61.190:36747
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.130
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:13:49.346302 23406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:13:49.348080 23406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:13:49.368695 23433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:13:49.368737 23435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:13:49.368943 23437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:13:49.370463 23406 server_base.cc:1034] running on GCE node
I20260811 19:13:49.371838 23406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:13:49.373620 23406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:13:49.375051 23406 hybrid_clock.cc:648] HybridClock initialized: now 1786475629374904 us; error 64 us; skew 500 ppm
I20260811 19:13:49.375664 23406 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:13:49 dist-test-slave-t7g5 krb5kdc[22326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475629, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:13:49.381705 23406 init.cc:377] Logged in from keytab as kudu/127.3.61.130@KRBTEST.COM (short username kudu)
I20260811 19:13:49.384393 23406 webserver.cc:460] Webserver started at http://127.3.61.130:39997/ using document root <none> and password file <none>
I20260811 19:13:49.385392 23406 fs_manager.cc:359] Metadata directory not provided
I20260811 19:13:49.385557 23406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:13:49.385995 23406 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 19:13:49 dist-test-slave-t7g5 krb5kdc[22326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475629, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.130@KRBTEST.COM for HTTP/127.3.61.148@KRBTEST.COM
I20260811 19:13:49.405558 23406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ts-1/data/instance:
uuid: "4a253a902a3340cfa089e032e75f3074"
format_stamp: "Formatted at 2026-08-11 19:13:49 on dist-test-slave-t7g5"
server_key: "632f4c967fac343f1a989b8cf7b7ae90"
server_key_iv: "3a9e23c102fa149a864dba0ee8147373"
server_key_version: "kuduclusterkey@0"
I20260811 19:13:49.407359 23406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ts-1/wal/instance:
uuid: "4a253a902a3340cfa089e032e75f3074"
format_stamp: "Formatted at 2026-08-11 19:13:49 on dist-test-slave-t7g5"
server_key: "632f4c967fac343f1a989b8cf7b7ae90"
server_key_iv: "3a9e23c102fa149a864dba0ee8147373"
server_key_version: "kuduclusterkey@0"
I20260811 19:13:49.418181 23406 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.006s sys 0.005s
I20260811 19:13:49.435655 23444 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:13:49.437892 23406 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.002s
I20260811 19:13:49.438207 23406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ts-1/wal
uuid: "4a253a902a3340cfa089e032e75f3074"
format_stamp: "Formatted at 2026-08-11 19:13:49 on dist-test-slave-t7g5"
server_key: "632f4c967fac343f1a989b8cf7b7ae90"
server_key_iv: "3a9e23c102fa149a864dba0ee8147373"
server_key_version: "kuduclusterkey@0"
I20260811 19:13:49.438601 23406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:13:49.473678 23406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:13:49.481199 23406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:13:49.481945 23406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:13:49.483989 23406 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:13:49.486406 23406 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:13:49.486584 23406 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:13:49.486830 23406 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:13:49.486961 23406 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:13:49.518361 23400 heartbeater.cc:503] Master 127.3.61.190:36747 was elected leader, sending a full tablet report...
I20260811 19:13:49.593364 23406 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.130:41945
I20260811 19:13:49.593636 23615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.130:41945 every 8 connection(s)
I20260811 19:13:49.595130 23406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
I20260811 19:13:49.595762 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 23406
I20260811 19:13:49.595920 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ts-1/wal/instance
I20260811 19:13:49.596421 3318 external_mini_cluster.cc:1428] Setting key 490566bc55861e1530b2b1a6dd9d84ba
Aug 11 19:13:49 dist-test-slave-t7g5 krb5kdc[22326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475629, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.130@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:13:49.631711 23618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:13:49.600111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:43945 (local address 127.3.61.190:36747)
0811 19:13:49.604418 (+ 4307us) server_negotiation.cc:206] Beginning negotiation
0811 19:13:49.604426 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:13:49.604453 (+ 27us) server_negotiation.cc:407] Connection header received
0811 19:13:49.606791 (+ 2338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:13:49.606798 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:13:49.606989 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:13:49.607118 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:13:49.609683 (+ 2565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:13:49.611071 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:13:49.614026 (+ 2955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:13:49.614773 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:13:49.614893 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:13:49.622511 (+ 7618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:13:49.622518 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:13:49.622522 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:13:49.622594 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:13:49.627232 (+ 4638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:13:49.628283 (+ 1051us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:13:49.628287 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:13:49.628290 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:13:49.628545 (+ 255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:13:49.629164 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:13:49.629169 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:13:49.629171 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:13:49.629606 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:13:49.629831 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0811 19:13:49.630789 (+ 958us) server_negotiation.cc:299] Negotiation successful
0811 19:13:49.631071 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4124,"thread_start_us":3708,"threads_started":1}
I20260811 19:13:49.633065 23622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:13:49.600772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:36747 (local address 127.3.61.130:43945)
0811 19:13:49.602770 (+ 1998us) negotiation.cc:107] Waiting for socket to connect
0811 19:13:49.602830 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 19:13:49.606437 (+ 3607us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:13:49.607530 (+ 1093us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:13:49.607540 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:13:49.608384 (+ 844us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:13:49.609413 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:13:49.609441 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:13:49.611274 (+ 1833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:13:49.611278 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:13:49.613841 (+ 2563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:13:49.613863 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:13:49.615507 (+ 1644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:13:49.615527 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:13:49.616028 (+ 501us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:13:49.617332 (+ 1304us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:13:49.617411 (+ 79us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:13:49.622087 (+ 4676us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:13:49.627459 (+ 5372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:13:49.627470 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:13:49.627488 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:13:49.628127 (+ 639us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:13:49.628674 (+ 547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:13:49.628677 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:13:49.628680 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:13:49.629027 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:13:49.629770 (+ 743us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:13:49.629779 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:13:49.630325 (+ 546us) client_negotiation.cc:769] Sending connection context
0811 19:13:49.630654 (+ 329us) client_negotiation.cc:240] Negotiation successful
0811 19:13:49.631382 (+ 728us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1557,"thread_start_us":1090,"threads_started":1}
I20260811 19:13:49.635653 23616 heartbeater.cc:347] Connected to a master server at 127.3.61.190:36747
WARNING: no policy specified for kudu/127.3.61.131@KRBTEST.COM; defaulting to no policy
I20260811 19:13:49.636067 23616 heartbeater.cc:464] Registering TS with master...
I20260811 19:13:49.637282 23616 heartbeater.cc:511] Master 127.3.61.190:36747 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.131@KRBTEST.COM" created.
I20260811 19:13:49.639948 23073 ts_manager.cc:194] Registered new tserver with Master: 4a253a902a3340cfa089e032e75f3074 (127.3.61.130:41945)
I20260811 19:13:49.641254 23073 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.130@KRBTEST.COM'} at 127.3.61.130:43945
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab.
Entry for principal kudu/127.3.61.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab.
I20260811 19:13:49.678369 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.61.148:36437/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.131:0
--local_ip_for_outbound_sockets=127.3.61.131
--webserver_interface=127.3.61.131
--webserver_port=0
--tserver_master_addrs=127.3.61.190:36747
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab
--principal=kudu/127.3.61.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:36393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:13:50.482007 23626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:13:50.482445 23626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:13:50.482707 23626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:13:50.494906 23626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:13:50.495167 23626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:13:50.495446 23626 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.131
I20260811 19:13:50.512785 23626 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:36393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.3.61.148:36437/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab
--principal=kudu/127.3.61.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.61.131
--webserver_port=0
--tserver_master_addrs=127.3.61.190:36747
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.131
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:13:50.514663 23626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:13:50.516394 23626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:13:50.540695 23634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:13:50.541349 23635 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:13:50.541473 23639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:13:50.543318 23626 server_base.cc:1034] running on GCE node
I20260811 19:13:50.544232 23626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:13:50.545845 23626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:13:50.547222 23626 hybrid_clock.cc:648] HybridClock initialized: now 1786475630547075 us; error 49 us; skew 500 ppm
I20260811 19:13:50.547736 23626 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:13:50 dist-test-slave-t7g5 krb5kdc[22326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475630, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:13:50.554103 23626 init.cc:377] Logged in from keytab as kudu/127.3.61.131@KRBTEST.COM (short username kudu)
I20260811 19:13:50.556871 23626 webserver.cc:460] Webserver started at http://127.3.61.131:39725/ using document root <none> and password file <none>
I20260811 19:13:50.557905 23626 fs_manager.cc:359] Metadata directory not provided
I20260811 19:13:50.558172 23626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:13:50.558717 23626 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 19:13:50 dist-test-slave-t7g5 krb5kdc[22326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475630, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.131@KRBTEST.COM for HTTP/127.3.61.148@KRBTEST.COM
I20260811 19:13:50.582038 23626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ts-2/data/instance:
uuid: "1013b3f52f354a92bbe460b8393a2f79"
format_stamp: "Formatted at 2026-08-11 19:13:50 on dist-test-slave-t7g5"
server_key: "1dbb0e7a5705c5b3b8c15832ddd21b28"
server_key_iv: "ebccaeef2e53fc2a87bf58fb2145ddb7"
server_key_version: "kuduclusterkey@0"
I20260811 19:13:50.583652 23626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ts-2/wal/instance:
uuid: "1013b3f52f354a92bbe460b8393a2f79"
format_stamp: "Formatted at 2026-08-11 19:13:50 on dist-test-slave-t7g5"
server_key: "1dbb0e7a5705c5b3b8c15832ddd21b28"
server_key_iv: "ebccaeef2e53fc2a87bf58fb2145ddb7"
server_key_version: "kuduclusterkey@0"
I20260811 19:13:50.597754 23626 fs_manager.cc:691] Time spent creating directory manager: real 0.013s user 0.009s sys 0.005s
I20260811 19:13:50.621102 23646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:13:50.623406 23626 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.002s
I20260811 19:13:50.623737 23626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ts-2/wal
uuid: "1013b3f52f354a92bbe460b8393a2f79"
format_stamp: "Formatted at 2026-08-11 19:13:50 on dist-test-slave-t7g5"
server_key: "1dbb0e7a5705c5b3b8c15832ddd21b28"
server_key_iv: "ebccaeef2e53fc2a87bf58fb2145ddb7"
server_key_version: "kuduclusterkey@0"
I20260811 19:13:50.624166 23626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:13:50.645623 23616 heartbeater.cc:503] Master 127.3.61.190:36747 was elected leader, sending a full tablet report...
I20260811 19:13:50.652732 23626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:13:50.660550 23626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:13:50.661116 23626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:13:50.663038 23626 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:13:50.665601 23626 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:13:50.665740 23626 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260811 19:13:50.665940 23626 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:13:50.666085 23626 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:13:50.778534 23626 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.131:46619
I20260811 19:13:50.778638 23831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.131:46619 every 8 connection(s)
I20260811 19:13:50.780550 23626 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
I20260811 19:13:50.790442 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 23626
I20260811 19:13:50.790629 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ts-2/wal/instance
I20260811 19:13:50.791185 3318 external_mini_cluster.cc:1428] Setting key 379124507d2fef9992eb7218f7f83102
Aug 11 19:13:50 dist-test-slave-t7g5 krb5kdc[22326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475630, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.131@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:13:50.816169 23834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:13:50.785885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:59057 (local address 127.3.61.190:36747)
0811 19:13:50.787067 (+ 1182us) server_negotiation.cc:206] Beginning negotiation
0811 19:13:50.787072 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:13:50.788341 (+ 1269us) server_negotiation.cc:407] Connection header received
0811 19:13:50.791623 (+ 3282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:13:50.791630 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:13:50.791798 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:13:50.791943 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:13:50.794658 (+ 2715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:13:50.795916 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:13:50.799006 (+ 3090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:13:50.799664 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:13:50.799780 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:13:50.805531 (+ 5751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:13:50.805537 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:13:50.805540 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:13:50.805579 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:13:50.810431 (+ 4852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:13:50.811618 (+ 1187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:13:50.811624 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:13:50.811632 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:13:50.811890 (+ 258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:13:50.812940 (+ 1050us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:13:50.812948 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:13:50.812950 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:13:50.813540 (+ 590us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:13:50.813837 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0811 19:13:50.815174 (+ 1337us) server_negotiation.cc:299] Negotiation successful
0811 19:13:50.815491 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1070,"thread_start_us":823,"threads_started":1}
I20260811 19:13:50.817435 23835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:13:50.786668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:36747 (local address 127.3.61.131:59057)
0811 19:13:50.788152 (+ 1484us) negotiation.cc:107] Waiting for socket to connect
0811 19:13:50.788198 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 19:13:50.791266 (+ 3068us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:13:50.792304 (+ 1038us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:13:50.792315 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:13:50.793129 (+ 814us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:13:50.794416 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:13:50.794451 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:13:50.796183 (+ 1732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:13:50.796191 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:13:50.798755 (+ 2564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:13:50.798803 (+ 48us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:13:50.800048 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:13:50.800057 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:13:50.800322 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:13:50.801153 (+ 831us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:13:50.801181 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:13:50.805155 (+ 3974us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:13:50.810630 (+ 5475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:13:50.810640 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:13:50.810657 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:13:50.811453 (+ 796us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:13:50.812026 (+ 573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:13:50.812030 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:13:50.812033 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:13:50.812466 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:13:50.813860 (+ 1394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:13:50.813876 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:13:50.814444 (+ 568us) client_negotiation.cc:769] Sending connection context
0811 19:13:50.814993 (+ 549us) client_negotiation.cc:240] Negotiation successful
0811 19:13:50.815805 (+ 812us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1146,"thread_start_us":899,"threads_started":1}
I20260811 19:13:50.819552 23832 heartbeater.cc:347] Connected to a master server at 127.3.61.190:36747
I20260811 19:13:50.820016 23832 heartbeater.cc:464] Registering TS with master...
I20260811 19:13:50.821316 23832 heartbeater.cc:511] Master 127.3.61.190:36747 requested a full tablet report, sending...
I20260811 19:13:50.823696 23073 ts_manager.cc:194] Registered new tserver with Master: 1013b3f52f354a92bbe460b8393a2f79 (127.3.61.131:46619)
I20260811 19:13:50.825214 23073 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.131@KRBTEST.COM'} at 127.3.61.131:59057
I20260811 19:13:50.834335 3318 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 19:13:50.865747 23834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:13:50.841592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37032 (local address 127.3.61.190:36747)
0811 19:13:50.841903 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0811 19:13:50.841910 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:13:50.842777 (+ 867us) server_negotiation.cc:407] Connection header received
0811 19:13:50.842992 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:13:50.842996 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:13:50.843130 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:13:50.843224 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:13:50.846272 (+ 3048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:13:50.847742 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:13:50.850945 (+ 3203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:13:50.851669 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:13:50.851787 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:13:50.856743 (+ 4956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:13:50.856753 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:13:50.856758 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:13:50.856809 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:13:50.861147 (+ 4338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:13:50.862473 (+ 1326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:13:50.862478 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:13:50.862481 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:13:50.862721 (+ 240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:13:50.863368 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:13:50.863372 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:13:50.863374 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:13:50.863862 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:13:50.864110 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0811 19:13:50.864792 (+ 682us) server_negotiation.cc:299] Negotiation successful
0811 19:13:50.865231 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 19:13:50.871593 23073 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37032:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 19:13:50.884478 23073 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:13:50.939164 23850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:13:50.906212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40054 (local address 127.3.61.130:41945)
0811 19:13:50.921957 (+ 15745us) server_negotiation.cc:206] Beginning negotiation
0811 19:13:50.921970 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 19:13:50.923706 (+ 1736us) server_negotiation.cc:407] Connection header received
0811 19:13:50.923983 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:13:50.923990 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:13:50.924264 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:13:50.924518 (+ 254us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:13:50.927483 (+ 2965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:13:50.930077 (+ 2594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:13:50.935167 (+ 5090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:13:50.936895 (+ 1728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:13:50.937102 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:13:50.937483 (+ 381us) server_negotiation.cc:1035] Waiting for connection context
0811 19:13:50.938282 (+ 799us) server_negotiation.cc:299] Negotiation successful
0811 19:13:50.938548 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15406,"thread_start_us":16641,"threads_started":1}
I20260811 19:13:50.939577 23853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:13:50.906369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50644 (local address 127.3.61.131:46619)
0811 19:13:50.922957 (+ 16588us) server_negotiation.cc:206] Beginning negotiation
0811 19:13:50.922966 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:13:50.923025 (+ 59us) server_negotiation.cc:407] Connection header received
0811 19:13:50.923169 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:13:50.923178 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:13:50.923491 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:13:50.923825 (+ 334us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:13:50.926052 (+ 2227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:13:50.929093 (+ 3041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:13:50.935023 (+ 5930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:13:50.937564 (+ 2541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:13:50.937744 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:13:50.938123 (+ 379us) server_negotiation.cc:1035] Waiting for connection context
0811 19:13:50.938623 (+ 500us) server_negotiation.cc:299] Negotiation successful
0811 19:13:50.938821 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16338,"thread_start_us":18462,"threads_started":1}
I20260811 19:13:50.940019 23852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:13:50.905590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.130:41945 (local address 127.0.0.1:40054)
0811 19:13:50.923576 (+ 17986us) negotiation.cc:107] Waiting for socket to connect
0811 19:13:50.923594 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 19:13:50.923813 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:13:50.924533 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:13:50.924538 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:13:50.924570 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:13:50.927223 (+ 2653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:13:50.927241 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:13:50.930278 (+ 3037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:13:50.930284 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:13:50.935006 (+ 4722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:13:50.935019 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:13:50.937408 (+ 2389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:13:50.937419 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:13:50.937758 (+ 339us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:13:50.937831 (+ 73us) client_negotiation.cc:769] Sending connection context
0811 19:13:50.938026 (+ 195us) client_negotiation.cc:240] Negotiation successful
0811 19:13:50.938269 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17188,"mutex_wait_us":5,"thread_start_us":16583,"threads_started":1}
I20260811 19:13:50.940128 23847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:13:50.905559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.131:46619 (local address 127.0.0.1:50644)
0811 19:13:50.913982 (+ 8423us) negotiation.cc:107] Waiting for socket to connect
0811 19:13:50.914045 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 19:13:50.914325 (+ 280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:13:50.924009 (+ 9684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:13:50.924016 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:13:50.924046 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:13:50.925293 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:13:50.925358 (+ 65us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:13:50.929890 (+ 4532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:13:50.929896 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:13:50.934730 (+ 4834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:13:50.934748 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:13:50.937905 (+ 3157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:13:50.937912 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:13:50.938159 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:13:50.938192 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 19:13:50.938267 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 19:13:50.938395 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7898,"thread_start_us":8540,"threads_started":1}
I20260811 19:13:50.946234 23856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:13:50.920334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:43919 (local address 127.0.0.1:46976)
0811 19:13:50.928410 (+ 8076us) negotiation.cc:107] Waiting for socket to connect
0811 19:13:50.928451 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 19:13:50.928631 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:13:50.930554 (+ 1923us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:13:50.930559 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:13:50.930588 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:13:50.932806 (+ 2218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:13:50.932828 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:13:50.935941 (+ 3113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:13:50.935947 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:13:50.941219 (+ 5272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:13:50.941237 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:13:50.944698 (+ 3461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:13:50.944708 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:13:50.945098 (+ 390us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:13:50.945134 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 19:13:50.945241 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 19:13:50.945527 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7828,"thread_start_us":7424,"threads_started":1}
I20260811 19:13:50.946272 23855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:13:50.916724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46976 (local address 127.3.61.129:43919)
0811 19:13:50.929807 (+ 13083us) server_negotiation.cc:206] Beginning negotiation
0811 19:13:50.929817 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 19:13:50.929849 (+ 32us) server_negotiation.cc:407] Connection header received
0811 19:13:50.929959 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:13:50.929968 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:13:50.930316 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:13:50.930549 (+ 233us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:13:50.933085 (+ 2536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:13:50.935632 (+ 2547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:13:50.941439 (+ 5807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:13:50.944070 (+ 2631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:13:50.944271 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:13:50.944825 (+ 554us) server_negotiation.cc:1035] Waiting for connection context
0811 19:13:50.945340 (+ 515us) server_negotiation.cc:299] Negotiation successful
0811 19:13:50.945623 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12797,"thread_start_us":12530,"threads_started":1}
I20260811 19:13:50.945716 23515 tablet_service.cc:1467] Processing CreateTablet for tablet 9b07a5bb27524ba88ee208630596de24 (DEFAULT_TABLE table=test-table [id=39cd19be188848338c1c1c8dc1a50b2b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:13:50.945533 23729 tablet_service.cc:1467] Processing CreateTablet for tablet 9b07a5bb27524ba88ee208630596de24 (DEFAULT_TABLE table=test-table [id=39cd19be188848338c1c1c8dc1a50b2b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:13:50.948009 23729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b07a5bb27524ba88ee208630596de24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:13:50.948915 23515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b07a5bb27524ba88ee208630596de24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:13:50.953078 23294 tablet_service.cc:1467] Processing CreateTablet for tablet 9b07a5bb27524ba88ee208630596de24 (DEFAULT_TABLE table=test-table [id=39cd19be188848338c1c1c8dc1a50b2b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:13:50.955657 23294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b07a5bb27524ba88ee208630596de24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:13:50.968004 23857 tablet_bootstrap.cc:492] T 9b07a5bb27524ba88ee208630596de24 P 1013b3f52f354a92bbe460b8393a2f79: Bootstrap starting.
I20260811 19:13:50.973083 23859 tablet_bootstrap.cc:492] T 9b07a5bb27524ba88ee208630596de24 P 4a253a902a3340cfa089e032e75f3074: Bootstrap starting.
I20260811 19:13:50.974264 23857 tablet_bootstrap.cc:654] T 9b07a5bb27524ba88ee208630596de24 P 1013b3f52f354a92bbe460b8393a2f79: Neither blocks nor log segments found. Creating new log.
I20260811 19:13:50.976850 23857 log.cc:824] T 9b07a5bb27524ba88ee208630596de24 P 1013b3f52f354a92bbe460b8393a2f79: Log is configured to *not* fsync() on all Append() calls
I20260811 19:13:50.978583 23860 tablet_bootstrap.cc:492] T 9b07a5bb27524ba88ee208630596de24 P 087bd0e157ff44aebcc01a3617d8ddd5: Bootstrap starting.
I20260811 19:13:50.980273 23859 tablet_bootstrap.cc:654] T 9b07a5bb27524ba88ee208630596de24 P 4a253a902a3340cfa089e032e75f3074: Neither blocks nor log segments found. Creating new log.
I20260811 19:13:50.982638 23859 log.cc:824] T 9b07a5bb27524ba88ee208630596de24 P 4a253a902a3340cfa089e032e75f3074: Log is configured to *not* fsync() on all Append() calls
I20260811 19:13:50.984042 23860 tablet_bootstrap.cc:654] T 9b07a5bb27524ba88ee208630596de24 P 087bd0e157ff44aebcc01a3617d8ddd5: Neither blocks nor log segments found. Creating new log.
I20260811 19:13:50.985402 23857 tablet_bootstrap.cc:492] T 9b07a5bb27524ba88ee208630596de24 P 1013b3f52f354a92bbe460b8393a2f79: No bootstrap required, opened a new log
I20260811 19:13:50.985878 23857 ts_tablet_manager.cc:1397] T 9b07a5bb27524ba88ee208630596de24 P 1013b3f52f354a92bbe460b8393a2f79: Time spent bootstrapping tablet: real 0.018s user 0.008s sys 0.007s
I20260811 19:13:50.987402 23860 log.cc:824] T 9b07a5bb27524ba88ee208630596de24 P 087bd0e157ff44aebcc01a3617d8ddd5: Log is configured to *not* fsync() on all Append() calls
I20260811 19:13:50.989708 23859 tablet_bootstrap.cc:492] T 9b07a5bb27524ba88ee208630596de24 P 4a253a902a3340cfa089e032e75f3074: No bootstrap required, opened a new log
I20260811 19:13:50.990195 23859 ts_tablet_manager.cc:1397] T 9b07a5bb27524ba88ee208630596de24 P 4a253a902a3340cfa089e032e75f3074: Time spent bootstrapping tablet: real 0.017s user 0.007s sys 0.008s
I20260811 19:13:50.994128 23860 tablet_bootstrap.cc:492] T 9b07a5bb27524ba88ee208630596de24 P 087bd0e157ff44aebcc01a3617d8ddd5: No bootstrap required, opened a new log
I20260811 19:13:50.994591 23860 ts_tablet_manager.cc:1397] T 9b07a5bb27524ba88ee208630596de24 P 087bd0e157ff44aebcc01a3617d8ddd5: Time spent bootstrapping tablet: real 0.016s user 0.010s sys 0.003s
I20260811 19:13:51.002776 23857 raft_consensus.cc:348] T 9b07a5bb27524ba88ee208630596de24 P 1013b3f52f354a92bbe460b8393a2f79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a253a902a3340cfa089e032e75f3074" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41945 } } peers { permanent_uuid: "1013b3f52f354a92bbe460b8393a2f79" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 46619 } } peers { permanent_uuid: "087bd0e157ff44aebcc01a3617d8ddd5" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 43919 } }
I20260811 19:13:51.003527 23857 raft_consensus.cc:374] T 9b07a5bb27524ba88ee208630596de24 P 1013b3f52f354a92bbe460b8393a2f79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:13:51.003667 23857 raft_consensus.cc:729] T 9b07a5bb27524ba88ee208630596de24 P 1013b3f52f354a92bbe460b8393a2f79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1013b3f52f354a92bbe460b8393a2f79, State: Initialized, Role: FOLLOWER
I20260811 19:13:51.003727 23859 raft_consensus.cc:348] T 9b07a5bb27524ba88ee208630596de24 P 4a253a902a3340cfa089e032e75f3074 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a253a902a3340cfa089e032e75f3074" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41945 } } peers { permanent_uuid: "1013b3f52f354a92bbe460b8393a2f79" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 46619 } } peers { permanent_uuid: "087bd0e157ff44aebcc01a3617d8ddd5" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 43919 } }
I20260811 19:13:51.004823 23859 raft_consensus.cc:374] T 9b07a5bb27524ba88ee208630596de24 P 4a253a902a3340cfa089e032e75f3074 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:13:51.004709 23857 consensus_queue.cc:260] T 9b07a5bb27524ba88ee208630596de24 P 1013b3f52f354a92bbe460b8393a2f79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a253a902a3340cfa089e032e75f3074" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41945 } } peers { permanent_uuid: "1013b3f52f354a92bbe460b8393a2f79" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 46619 } } peers { permanent_uuid: "087bd0e157ff44aebcc01a3617d8ddd5" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 43919 } }
I20260811 19:13:51.005044 23859 raft_consensus.cc:729] T 9b07a5bb27524ba88ee208630596de24 P 4a253a902a3340cfa089e032e75f3074 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a253a902a3340cfa089e032e75f3074, State: Initialized, Role: FOLLOWER
I20260811 19:13:51.005959 23859 consensus_queue.cc:260] T 9b07a5bb27524ba88ee208630596de24 P 4a253a902a3340cfa089e032e75f3074 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a253a902a3340cfa089e032e75f3074" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41945 } } peers { permanent_uuid: "1013b3f52f354a92bbe460b8393a2f79" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 46619 } } peers { permanent_uuid: "087bd0e157ff44aebcc01a3617d8ddd5" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 43919 } }
I20260811 19:13:51.006855 23832 heartbeater.cc:503] Master 127.3.61.190:36747 was elected leader, sending a full tablet report...
I20260811 19:13:51.007891 23857 ts_tablet_manager.cc:1428] T 9b07a5bb27524ba88ee208630596de24 P 1013b3f52f354a92bbe460b8393a2f79: Time spent starting tablet: real 0.022s user 0.018s sys 0.003s
I20260811 19:13:51.010161 23860 raft_consensus.cc:348] T 9b07a5bb27524ba88ee208630596de24 P 087bd0e157ff44aebcc01a3617d8ddd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a253a902a3340cfa089e032e75f3074" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41945 } } peers { permanent_uuid: "1013b3f52f354a92bbe460b8393a2f79" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 46619 } } peers { permanent_uuid: "087bd0e157ff44aebcc01a3617d8ddd5" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 43919 } }
I20260811 19:13:51.011015 23860 raft_consensus.cc:374] T 9b07a5bb27524ba88ee208630596de24 P 087bd0e157ff44aebcc01a3617d8ddd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:13:51.011264 23860 raft_consensus.cc:729] T 9b07a5bb27524ba88ee208630596de24 P 087bd0e157ff44aebcc01a3617d8ddd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 087bd0e157ff44aebcc01a3617d8ddd5, State: Initialized, Role: FOLLOWER
I20260811 19:13:51.011437 23859 ts_tablet_manager.cc:1428] T 9b07a5bb27524ba88ee208630596de24 P 4a253a902a3340cfa089e032e75f3074: Time spent starting tablet: real 0.021s user 0.010s sys 0.012s
I20260811 19:13:51.012297 23860 consensus_queue.cc:260] T 9b07a5bb27524ba88ee208630596de24 P 087bd0e157ff44aebcc01a3617d8ddd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a253a902a3340cfa089e032e75f3074" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41945 } } peers { permanent_uuid: "1013b3f52f354a92bbe460b8393a2f79" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 46619 } } peers { permanent_uuid: "087bd0e157ff44aebcc01a3617d8ddd5" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 43919 } }
I20260811 19:13:51.015807 23860 ts_tablet_manager.cc:1428] T 9b07a5bb27524ba88ee208630596de24 P 087bd0e157ff44aebcc01a3617d8ddd5: Time spent starting tablet: real 0.021s user 0.020s sys 0.000s
W20260811 19:13:51.034798 23833 tablet.cc:2369] T 9b07a5bb27524ba88ee208630596de24 P 1013b3f52f354a92bbe460b8393a2f79: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:13:51.093371 23868 raft_consensus.cc:482] T 9b07a5bb27524ba88ee208630596de24 P 4a253a902a3340cfa089e032e75f3074 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:13:51.093927 23868 raft_consensus.cc:504] T 9b07a5bb27524ba88ee208630596de24 P 4a253a902a3340cfa089e032e75f3074 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a253a902a3340cfa089e032e75f3074" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41945 } } peers { permanent_uuid: "1013b3f52f354a92bbe460b8393a2f79" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 46619 } } peers { permanent_uuid: "087bd0e157ff44aebcc01a3617d8ddd5" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 43919 } }
I20260811 19:13:51.096712 23868 leader_election.cc:288] T 9b07a5bb27524ba88ee208630596de24 P 4a253a902a3340cfa089e032e75f3074 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1013b3f52f354a92bbe460b8393a2f79 (127.3.61.131:46619), 087bd0e157ff44aebcc01a3617d8ddd5 (127.3.61.129:43919)
W20260811 19:13:51.100255 23617 tablet.cc:2369] T 9b07a5bb27524ba88ee208630596de24 P 4a253a902a3340cfa089e032e75f3074: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:13:51.108862 23853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:13:51.097350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:34397 (local address 127.3.61.131:46619)
0811 19:13:51.097747 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0811 19:13:51.097755 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:13:51.098770 (+ 1015us) server_negotiation.cc:407] Connection header received
0811 19:13:51.099027 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:13:51.099031 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:13:51.099106 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:13:51.099198 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:13:51.100842 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:13:51.102584 (+ 1742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:13:51.106322 (+ 3738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:13:51.107754 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:13:51.107931 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:13:51.108015 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 19:13:51.108426 (+ 411us) server_negotiation.cc:299] Negotiation successful
0811 19:13:51.108564 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
I20260811 19:13:51.108981 23872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:13:51.097127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.131:46619 (local address 127.3.61.130:34397)
0811 19:13:51.098623 (+ 1496us) negotiation.cc:107] Waiting for socket to connect
0811 19:13:51.098641 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 19:13:51.098853 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:13:51.099378 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:13:51.099383 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:13:51.099412 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:13:51.100603 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:13:51.100623 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:13:51.102775 (+ 2152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:13:51.102779 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:13:51.106070 (+ 3291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:13:51.106087 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:13:51.108107 (+ 2020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:13:51.108112 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:13:51.108261 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:13:51.108289 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 19:13:51.108377 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 19:13:51.108532 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1332,"thread_start_us":836,"threads_started":1}
I20260811 19:13:51.110430 23873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:13:51.098836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:43919 (local address 127.3.61.130:47469)
0811 19:13:51.100563 (+ 1727us) negotiation.cc:107] Waiting for socket to connect
0811 19:13:51.100585 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 19:13:51.100822 (+ 237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:13:51.101214 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:13:51.101219 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:13:51.101246 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:13:51.102680 (+ 1434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:13:51.102696 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:13:51.104362 (+ 1666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:13:51.104368 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:13:51.107594 (+ 3226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:13:51.107610 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:13:51.109648 (+ 2038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:13:51.109681 (+ 33us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:13:51.109842 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:13:51.109870 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 19:13:51.109953 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 19:13:51.110055 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1501,"thread_start_us":1013,"threads_started":1}
I20260811 19:13:51.110597 23855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:13:51.099014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:47469 (local address 127.3.61.129:43919)
0811 19:13:51.099274 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0811 19:13:51.099281 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:13:51.100674 (+ 1393us) server_negotiation.cc:407] Connection header received
0811 19:13:51.100992 (+ 318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:13:51.100998 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:13:51.101081 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:13:51.101182 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:13:51.102925 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:13:51.104157 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:13:51.107772 (+ 3615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:13:51.109318 (+ 1546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:13:51.109721 (+ 403us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:13:51.109828 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 19:13:51.110024 (+ 196us) server_negotiation.cc:299] Negotiation successful
0811 19:13:51.110205 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 19:13:51.111048 23763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b07a5bb27524ba88ee208630596de24" candidate_uuid: "4a253a902a3340cfa089e032e75f3074" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1013b3f52f354a92bbe460b8393a2f79" is_pre_election: true
I20260811 19:13:51.111708 23763 raft_consensus.cc:2454] T 9b07a5bb27524ba88ee208630596de24 P 1013b3f52f354a92bbe460b8393a2f79 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a253a902a3340cfa089e032e75f3074 in term 0.
I20260811 19:13:51.112689 23448 leader_election.cc:302] T 9b07a5bb27524ba88ee208630596de24 P 4a253a902a3340cfa089e032e75f3074 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1013b3f52f354a92bbe460b8393a2f79, 4a253a902a3340cfa089e032e75f3074; no voters:
I20260811 19:13:51.112640 23325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b07a5bb27524ba88ee208630596de24" candidate_uuid: "4a253a902a3340cfa089e032e75f3074" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "087bd0e157ff44aebcc01a3617d8ddd5" is_pre_election: true
I20260811 19:13:51.113507 23868 raft_consensus.cc:2781] T 9b07a5bb27524ba88ee208630596de24 P 4a253a902a3340cfa089e032e75f3074 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:13:51.113719 23868 raft_consensus.cc:482] T 9b07a5bb27524ba88ee208630596de24 P 4a253a902a3340cfa089e032e75f3074 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:13:51.113845 23868 raft_consensus.cc:3037] T 9b07a5bb27524ba88ee208630596de24 P 4a253a902a3340cfa089e032e75f3074 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:13:51.113879 23325 raft_consensus.cc:2454] T 9b07a5bb27524ba88ee208630596de24 P 087bd0e157ff44aebcc01a3617d8ddd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a253a902a3340cfa089e032e75f3074 in term 0.
I20260811 19:13:51.117856 23868 raft_consensus.cc:504] T 9b07a5bb27524ba88ee208630596de24 P 4a253a902a3340cfa089e032e75f3074 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a253a902a3340cfa089e032e75f3074" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41945 } } peers { permanent_uuid: "1013b3f52f354a92bbe460b8393a2f79" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 46619 } } peers { permanent_uuid: "087bd0e157ff44aebcc01a3617d8ddd5" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 43919 } }
I20260811 19:13:51.119047 23868 leader_election.cc:288] T 9b07a5bb27524ba88ee208630596de24 P 4a253a902a3340cfa089e032e75f3074 [CANDIDATE]: Term 1 election: Requested vote from peers 1013b3f52f354a92bbe460b8393a2f79 (127.3.61.131:46619), 087bd0e157ff44aebcc01a3617d8ddd5 (127.3.61.129:43919)
I20260811 19:13:51.119705 23325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b07a5bb27524ba88ee208630596de24" candidate_uuid: "4a253a902a3340cfa089e032e75f3074" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "087bd0e157ff44aebcc01a3617d8ddd5"
I20260811 19:13:51.119802 23763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b07a5bb27524ba88ee208630596de24" candidate_uuid: "4a253a902a3340cfa089e032e75f3074" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1013b3f52f354a92bbe460b8393a2f79"
I20260811 19:13:51.120030 23325 raft_consensus.cc:3037] T 9b07a5bb27524ba88ee208630596de24 P 087bd0e157ff44aebcc01a3617d8ddd5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:13:51.120079 23763 raft_consensus.cc:3037] T 9b07a5bb27524ba88ee208630596de24 P 1013b3f52f354a92bbe460b8393a2f79 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:13:51.123660 23763 raft_consensus.cc:2454] T 9b07a5bb27524ba88ee208630596de24 P 1013b3f52f354a92bbe460b8393a2f79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a253a902a3340cfa089e032e75f3074 in term 1.
I20260811 19:13:51.124009 23325 raft_consensus.cc:2454] T 9b07a5bb27524ba88ee208630596de24 P 087bd0e157ff44aebcc01a3617d8ddd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a253a902a3340cfa089e032e75f3074 in term 1.
I20260811 19:13:51.124225 23448 leader_election.cc:302] T 9b07a5bb27524ba88ee208630596de24 P 4a253a902a3340cfa089e032e75f3074 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1013b3f52f354a92bbe460b8393a2f79, 4a253a902a3340cfa089e032e75f3074; no voters:
I20260811 19:13:51.124796 23868 raft_consensus.cc:2781] T 9b07a5bb27524ba88ee208630596de24 P 4a253a902a3340cfa089e032e75f3074 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:13:51.125773 23868 raft_consensus.cc:686] T 9b07a5bb27524ba88ee208630596de24 P 4a253a902a3340cfa089e032e75f3074 [term 1 LEADER]: Becoming Leader. State: Replica: 4a253a902a3340cfa089e032e75f3074, State: Running, Role: LEADER
I20260811 19:13:51.126616 23868 consensus_queue.cc:237] T 9b07a5bb27524ba88ee208630596de24 P 4a253a902a3340cfa089e032e75f3074 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a253a902a3340cfa089e032e75f3074" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41945 } } peers { permanent_uuid: "1013b3f52f354a92bbe460b8393a2f79" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 46619 } } peers { permanent_uuid: "087bd0e157ff44aebcc01a3617d8ddd5" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 43919 } }
I20260811 19:13:51.133724 23072 catalog_manager.cc:5495] T 9b07a5bb27524ba88ee208630596de24 P 4a253a902a3340cfa089e032e75f3074 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4a253a902a3340cfa089e032e75f3074 (127.3.61.130). New cstate: current_term: 1 leader_uuid: "4a253a902a3340cfa089e032e75f3074" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a253a902a3340cfa089e032e75f3074" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41945 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1013b3f52f354a92bbe460b8393a2f79" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 46619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "087bd0e157ff44aebcc01a3617d8ddd5" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 43919 } health_report { overall_health: UNKNOWN } } }
I20260811 19:13:51.177019 23850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:13:51.166924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40068 (local address 127.3.61.130:41945)
0811 19:13:51.167326 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0811 19:13:51.167333 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:13:51.167365 (+ 32us) server_negotiation.cc:407] Connection header received
0811 19:13:51.167706 (+ 341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:13:51.167713 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:13:51.167867 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:13:51.168024 (+ 157us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:13:51.169075 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:13:51.170715 (+ 1640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:13:51.173220 (+ 2505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:13:51.174158 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:13:51.174297 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:13:51.175402 (+ 1105us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:13:51.176082 (+ 680us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:13:51.176209 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 19:13:51.176579 (+ 370us) server_negotiation.cc:299] Negotiation successful
0811 19:13:51.176671 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128}
I20260811 19:13:51.189805 23325 raft_consensus.cc:1261] T 9b07a5bb27524ba88ee208630596de24 P 087bd0e157ff44aebcc01a3617d8ddd5 [term 1 FOLLOWER]: Refusing update from remote peer 4a253a902a3340cfa089e032e75f3074: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:13:51.190003 23763 raft_consensus.cc:1261] T 9b07a5bb27524ba88ee208630596de24 P 1013b3f52f354a92bbe460b8393a2f79 [term 1 FOLLOWER]: Refusing update from remote peer 4a253a902a3340cfa089e032e75f3074: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:13:51.191167 23876 consensus_queue.cc:1035] T 9b07a5bb27524ba88ee208630596de24 P 4a253a902a3340cfa089e032e75f3074 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1013b3f52f354a92bbe460b8393a2f79" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 46619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260811 19:13:51.192281 23868 consensus_queue.cc:1035] T 9b07a5bb27524ba88ee208630596de24 P 4a253a902a3340cfa089e032e75f3074 [LEADER]: Connected to new peer: Peer: permanent_uuid: "087bd0e157ff44aebcc01a3617d8ddd5" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 43919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:13:51.211076 23883 mvcc.cc:205] Tried to move back new op lower bound from 7317404185341276160 to 7317404185099886592. Current Snapshot: MvccSnapshot[applied={T|T < 7317404185341276160}]
I20260811 19:13:51.235669 23073 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37032:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:13:51.236227 23073 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37032:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:13:51.241966 23073 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e3d4b43916084f5bb2d4bf0041873fde: Sending DeleteTablet for 3 replicas of tablet 9b07a5bb27524ba88ee208630596de24
I20260811 19:13:51.243834 23515 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b07a5bb27524ba88ee208630596de24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:13:51 UTC) from {username='kudu', principal='kudu/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:40054
I20260811 19:13:51.244264 23294 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b07a5bb27524ba88ee208630596de24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:13:51 UTC) from {username='kudu', principal='kudu/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:46976
I20260811 19:13:51.244962 23729 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b07a5bb27524ba88ee208630596de24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:13:51 UTC) from {username='kudu', principal='kudu/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:50644
I20260811 19:13:51.246227 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 23200
I20260811 19:13:51.258227 23905 tablet_replica.cc:330] T 9b07a5bb27524ba88ee208630596de24 P 1013b3f52f354a92bbe460b8393a2f79: stopping tablet replica
I20260811 19:13:51.258831 23905 raft_consensus.cc:2229] T 9b07a5bb27524ba88ee208630596de24 P 1013b3f52f354a92bbe460b8393a2f79 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:13:51.259467 23905 raft_consensus.cc:2258] T 9b07a5bb27524ba88ee208630596de24 P 1013b3f52f354a92bbe460b8393a2f79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:13:51.260704 23904 tablet_replica.cc:330] T 9b07a5bb27524ba88ee208630596de24 P 087bd0e157ff44aebcc01a3617d8ddd5: stopping tablet replica
I20260811 19:13:51.261189 23904 raft_consensus.cc:2229] T 9b07a5bb27524ba88ee208630596de24 P 087bd0e157ff44aebcc01a3617d8ddd5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:13:51.261917 23904 raft_consensus.cc:2258] T 9b07a5bb27524ba88ee208630596de24 P 087bd0e157ff44aebcc01a3617d8ddd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:13:51.262674 23905 ts_tablet_manager.cc:1905] T 9b07a5bb27524ba88ee208630596de24 P 1013b3f52f354a92bbe460b8393a2f79: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:13:51.264348 23904 ts_tablet_manager.cc:1905] T 9b07a5bb27524ba88ee208630596de24 P 087bd0e157ff44aebcc01a3617d8ddd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:13:51.269932 23905 ts_tablet_manager.cc:1918] T 9b07a5bb27524ba88ee208630596de24 P 1013b3f52f354a92bbe460b8393a2f79: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:13:51.270138 23905 log.cc:1196] T 9b07a5bb27524ba88ee208630596de24 P 1013b3f52f354a92bbe460b8393a2f79: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ts-2/wal/wals/9b07a5bb27524ba88ee208630596de24
I20260811 19:13:51.270668 23905 ts_tablet_manager.cc:1939] T 9b07a5bb27524ba88ee208630596de24 P 1013b3f52f354a92bbe460b8393a2f79: Deleting consensus metadata
I20260811 19:13:51.272855 23050 catalog_manager.cc:4841] TS 1013b3f52f354a92bbe460b8393a2f79 (127.3.61.131:46619): tablet 9b07a5bb27524ba88ee208630596de24 (table test-table [id=39cd19be188848338c1c1c8dc1a50b2b]) successfully deleted
I20260811 19:13:51.273674 23904 ts_tablet_manager.cc:1918] T 9b07a5bb27524ba88ee208630596de24 P 087bd0e157ff44aebcc01a3617d8ddd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:13:51.273787 23902 tablet_replica.cc:330] T 9b07a5bb27524ba88ee208630596de24 P 4a253a902a3340cfa089e032e75f3074: stopping tablet replica
I20260811 19:13:51.273942 23904 log.cc:1196] T 9b07a5bb27524ba88ee208630596de24 P 087bd0e157ff44aebcc01a3617d8ddd5: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ts-0/wal/wals/9b07a5bb27524ba88ee208630596de24
I20260811 19:13:51.274325 23902 raft_consensus.cc:2229] T 9b07a5bb27524ba88ee208630596de24 P 4a253a902a3340cfa089e032e75f3074 [term 1 LEADER]: Raft consensus shutting down.
I20260811 19:13:51.274752 23904 ts_tablet_manager.cc:1939] T 9b07a5bb27524ba88ee208630596de24 P 087bd0e157ff44aebcc01a3617d8ddd5: Deleting consensus metadata
I20260811 19:13:51.275166 23902 raft_consensus.cc:2258] T 9b07a5bb27524ba88ee208630596de24 P 4a253a902a3340cfa089e032e75f3074 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:13:51.277160 23050 catalog_manager.cc:4841] TS 087bd0e157ff44aebcc01a3617d8ddd5 (127.3.61.129:43919): tablet 9b07a5bb27524ba88ee208630596de24 (table test-table [id=39cd19be188848338c1c1c8dc1a50b2b]) successfully deleted
Aug 11 19:13:51 dist-test-slave-t7g5 krb5kdc[22326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475625, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.129@KRBTEST.COM
I20260811 19:13:51.278208 23902 ts_tablet_manager.cc:1905] T 9b07a5bb27524ba88ee208630596de24 P 4a253a902a3340cfa089e032e75f3074: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:13:51.286749 23902 ts_tablet_manager.cc:1918] T 9b07a5bb27524ba88ee208630596de24 P 4a253a902a3340cfa089e032e75f3074: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:13:51.286959 23902 log.cc:1196] T 9b07a5bb27524ba88ee208630596de24 P 4a253a902a3340cfa089e032e75f3074: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786475454804547-3318-0/minicluster-data/ts-1/wal/wals/9b07a5bb27524ba88ee208630596de24
I20260811 19:13:51.287468 23902 ts_tablet_manager.cc:1939] T 9b07a5bb27524ba88ee208630596de24 P 4a253a902a3340cfa089e032e75f3074: Deleting consensus metadata
I20260811 19:13:51.289762 23044 catalog_manager.cc:4841] TS 4a253a902a3340cfa089e032e75f3074 (127.3.61.130:41945): tablet 9b07a5bb27524ba88ee208630596de24 (table test-table [id=39cd19be188848338c1c1c8dc1a50b2b]) successfully deleted
I20260811 19:13:51.290759 23855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:13:51.247785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46992 (local address 127.3.61.129:43919)
0811 19:13:51.247995 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 19:13:51.248001 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:13:51.257179 (+ 9178us) server_negotiation.cc:407] Connection header received
0811 19:13:51.257431 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:13:51.257437 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:13:51.257586 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:13:51.257754 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:13:51.263883 (+ 6129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:13:51.265053 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:13:51.269705 (+ 4652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:13:51.270569 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:13:51.270688 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:13:51.279509 (+ 8821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:13:51.279533 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:13:51.279552 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:13:51.279660 (+ 108us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:13:51.285931 (+ 6271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:13:51.287373 (+ 1442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:13:51.287385 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:13:51.287388 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:13:51.287620 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:13:51.288279 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:13:51.288292 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:13:51.288296 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:13:51.289049 (+ 753us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:13:51.289551 (+ 502us) server_negotiation.cc:1035] Waiting for connection context
0811 19:13:51.289956 (+ 405us) server_negotiation.cc:299] Negotiation successful
0811 19:13:51.290182 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 19:13:51.291850 23387 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:13:51.392323 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 23200
I20260811 19:13:51.428439 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 23406
Aug 11 19:13:51 dist-test-slave-t7g5 krb5kdc[22326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475625, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.130@KRBTEST.COM
I20260811 19:13:51.457286 23850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:13:51.429418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40074 (local address 127.3.61.130:41945)
0811 19:13:51.429596 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 19:13:51.429600 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:13:51.429727 (+ 127us) server_negotiation.cc:407] Connection header received
0811 19:13:51.430019 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:13:51.430024 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:13:51.430111 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:13:51.430204 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:13:51.432012 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:13:51.433497 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:13:51.435813 (+ 2316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:13:51.436820 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:13:51.436982 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:13:51.448056 (+ 11074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:13:51.448081 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:13:51.448105 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:13:51.448167 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:13:51.453213 (+ 5046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:13:51.454380 (+ 1167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:13:51.454390 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:13:51.454393 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:13:51.454694 (+ 301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:13:51.455214 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:13:51.455223 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:13:51.455225 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:13:51.455763 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:13:51.456092 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0811 19:13:51.456619 (+ 527us) server_negotiation.cc:299] Negotiation successful
0811 19:13:51.456884 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 19:13:51.458184 23611 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:13:51.576481 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 23406
I20260811 19:13:51.602854 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 23626
Aug 11 19:13:51 dist-test-slave-t7g5 krb5kdc[22326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475625, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.131@KRBTEST.COM
I20260811 19:13:51.621069 23853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:13:51.603674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50654 (local address 127.3.61.131:46619)
0811 19:13:51.603837 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 19:13:51.603842 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:13:51.603879 (+ 37us) server_negotiation.cc:407] Connection header received
0811 19:13:51.604035 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:13:51.604045 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:13:51.604116 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:13:51.604204 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:13:51.606022 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:13:51.606795 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:13:51.608240 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:13:51.608807 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:13:51.608977 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:13:51.613857 (+ 4880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:13:51.613880 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:13:51.613899 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:13:51.613956 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:13:51.617265 (+ 3309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:13:51.618374 (+ 1109us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:13:51.618388 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:13:51.618390 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:13:51.618716 (+ 326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:13:51.619306 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:13:51.619312 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:13:51.619314 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:13:51.619856 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:13:51.620156 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0811 19:13:51.620545 (+ 389us) server_negotiation.cc:299] Negotiation successful
0811 19:13:51.620750 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 19:13:51.621735 23824 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:13:51.717353 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 23626
I20260811 19:13:51.740799 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 23020
I20260811 19:13:51.742132 23126 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:13:51.856626 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 23020
I20260811 19:13:51.894317 3318 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 19:13:52.242095 3318 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 19:13:52.242443 3318 mini_ranger.cc:66] Stopping Ranger...
I20260811 19:13:52.600648 3318 mini_ranger.cc:68] Stopped Ranger
2026-08-11 19:13:52.601 UTC [22376] LOG: received smart shutdown request
2026-08-11 19:13:52.605 UTC [22376] LOG: background worker "logical replication launcher" (PID 22389) exited with exit code 1
2026-08-11 19:13:52.605 UTC [22381] LOG: shutting down
2026-08-11 19:13:52.774 UTC [22376] LOG: database system is shut down
2026-08-11T19:13:52Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (60928 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:13:53 dist-test-slave-t7g5 krb5kdc[23924](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:13:53 dist-test-slave-t7g5 krb5kdc[23924](info): set up 2 sockets
Aug 11 19:13:53 dist-test-slave-t7g5 krb5kdc[23924](info): commencing operation
krb5kdc: starting...
W20260811 19:13:54.225939 3318 mini_kdc.cc:121] Time spent starting KDC: real 1.277s user 0.000s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Aug 11 19:13:54 dist-test-slave-t7g5 krb5kdc[23924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475634, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:13:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:13:54Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.3.61.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.3.61.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.3.61.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/krb5kdc/rangeradmin_127.3.61.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.3.61.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/krb5kdc/rangeradmin_127.3.61.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.3.61.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.3.61.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.3.61.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/krb5kdc/rangerlookup_127.3.61.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.3.61.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/krb5kdc/rangerlookup_127.3.61.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.3.61.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.3.61.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.3.61.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/krb5kdc/HTTP_127.3.61.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.3.61.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/krb5kdc/HTTP_127.3.61.148@KRBTEST.COM.keytab.
I20260811 19:13:54.710078 3318 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskwEqXVd/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/postgres -l logfile start
2026-08-11 19:13:57.467 UTC [23977] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 19:13:57.467 UTC [23977] LOG: listening on IPv4 address "127.3.61.148", port 60373
2026-08-11 19:13:57.471 UTC [23977] LOG: listening on Unix socket "/tmp/.s.PGSQL.60373"
2026-08-11 19:13:57.497 UTC [23979] LOG: database system was shut down at 2026-08-11 19:13:55 UTC
2026-08-11 19:13:57.501 UTC [23977] LOG: database system is ready to accept connections
I20260811 19:13:58.654326 3318 mini_postgres.cc:96] Postgres bound to 60373
2026-08-11 19:13:58.680 UTC [23997] FATAL: database "slave" does not exist
127.3.61.148:60373 - accepting connections
I20260811 19:13:58.681596 3318 mini_ranger.cc:161] Starting Ranger...
I20260811 19:13:58.718041 3318 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 19:13:59.265080 3318 mini_ranger.cc:87] Created ranger Postgres database
I20260811 19:13:59.265224 3318 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ranger-admin
2026-08-11 19:13:59,583 [I] DB FLAVOR :POSTGRES
2026-08-11 19:13:59,584 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 19:13:59,584 [I] Checking connection..
2026-08-11 19:13:59,584 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwEqXVd/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwEqXVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.61.148:60373/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 19:13:59,878 [I] Checking connection passed.
2026-08-11 19:13:59,878 [I] --------- Verifying version history table ---------
2026-08-11 19:13:59,878 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwEqXVd/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwEqXVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.61.148:60373/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 19:14:00,174 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 19:14:00,174 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwEqXVd/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwEqXVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.61.148:60373/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 19:14:00,457 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 19:14:00,457 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 19:14:00,458 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwEqXVd/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwEqXVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.61.148:60373/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskwEqXVd/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 19:14:00.732 UTC [24093] WARNING: there is no transaction in progress
2026-08-11 19:14:00,742 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 19:14:00,742 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwEqXVd/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwEqXVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.61.148:60373/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 19:14:01,035 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 19:14:01,036 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 19:14:01,036 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwEqXVd/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwEqXVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.61.148:60373/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 19:14:01,314 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwEqXVd/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwEqXVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.61.148:60373/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 19:14:01,607 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwEqXVd/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwEqXVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.61.148:60373/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-t7g5.c.gcp-upstream.internal','N') ;"
2026-08-11 19:14:01,888 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwEqXVd/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwEqXVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.61.148:60373/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 19:14:02,175 [I] Table x_portal_user does not exist in database ranger
2026-08-11 19:14:02,175 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwEqXVd/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwEqXVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.61.148:60373/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 19:14:02,482 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 19:14:02,482 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwEqXVd/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwEqXVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.61.148:60373/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 19:14:02,777 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwEqXVd/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwEqXVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.61.148:60373/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 19:14:03,074 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 19:14:03,074 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwEqXVd/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwEqXVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.61.148:60373/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskwEqXVd/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 19:14:04.228 UTC [24267] WARNING: there is no transaction in progress
2026-08-11 19:14:04.248 UTC [24267] WARNING: there is no transaction in progress
2026-08-11 19:14:04.270 UTC [24267] WARNING: there is no transaction in progress
2026-08-11 19:14:04.288 UTC [24267] WARNING: there is no transaction in progress
2026-08-11 19:14:04.307 UTC [24267] WARNING: there is no transaction in progress
2026-08-11 19:14:04.326 UTC [24267] WARNING: there is no transaction in progress
2026-08-11 19:14:04.392 UTC [24267] WARNING: there is no transaction in progress
2026-08-11 19:14:04.400 UTC [24267] WARNING: there is no transaction in progress
2026-08-11 19:14:04.416 UTC [24267] WARNING: there is no transaction in progress
2026-08-11 19:14:04.429 UTC [24267] WARNING: there is no transaction in progress
2026-08-11 19:14:04.440 UTC [24267] WARNING: there is no transaction in progress
2026-08-11 19:14:04.457 UTC [24267] WARNING: there is no transaction in progress
2026-08-11 19:14:04.468 UTC [24267] WARNING: there is no transaction in progress
2026-08-11 19:14:04.476 UTC [24267] WARNING: there is no transaction in progress
2026-08-11 19:14:04.485 UTC [24267] WARNING: there is no transaction in progress
2026-08-11 19:14:04.494 UTC [24267] WARNING: there is no transaction in progress
2026-08-11 19:14:05,314 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 19:14:05,315 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwEqXVd/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwEqXVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.61.148:60373/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 19:14:05,600 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 19:14:05,601 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwEqXVd/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwEqXVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.61.148:60373/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 19:14:05,878 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 19:14:05,879 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwEqXVd/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwEqXVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.61.148:60373/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 19:14:06,178 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 19:14:06,179 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwEqXVd/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwEqXVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.61.148:60373/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 19:14:06,468 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwEqXVd/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwEqXVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.61.148:60373/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 19:14:06,756 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwEqXVd/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwEqXVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.61.148:60373/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-t7g5.c.gcp-upstream.internal';"
2026-08-11 19:14:07,038 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 19:14:07,038 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwEqXVd/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwEqXVd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.61.148:60373/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 19:14:07,316 [I] DB_PATCHES have already been applied
I20260811 19:14:07.322755 3318 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ranger-admin:/tmp/dist-test-taskwEqXVd/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskwEqXVd/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskwEqXVd/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwEqXVd/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 19:14:07.322866 3318 mini_ranger.cc:193] Using host: 127.3.61.148
I20260811 19:14:07.345880 3318 mini_ranger.cc:227] Ranger admin URL: http://127.3.61.148:53373
Aug 11, 2026 7:14:07 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 7:14:07 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 7:14:07 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 7:14:07 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 7:14:07 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 7:14:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 7:14:08 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 19:14:08.720964 3318 mini_ranger.cc:160] Time spent starting Ranger: real 10.039s user 0.006s sys 0.078s
Aug 11, 2026 7:14:30 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 7:14:30 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 11, 2026 7:14:30 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 7:14:30 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 7:14:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 7:14:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 7:14:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 7:14:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 7:14:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 7:14:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 7:14:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 7:14:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 7:14:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 7:14:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 7:14:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 7:14:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 7:14:30 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 7:14:31 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 19:14:32.221843 3318 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.3.61.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.3.61.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.3.61.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/krb5kdc/rangerkms_127.3.61.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.3.61.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/krb5kdc/rangerkms_127.3.61.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.3.61.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.3.61.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.3.61.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/krb5kdc/HTTP_127.3.61.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.3.61.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/krb5kdc/HTTP_127.3.61.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 19:14:32 dist-test-slave-t7g5 krb5kdc[23924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475672, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 19:14:32.480858 3318 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 19:14:32.512815 3318 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 19:14:33.420728 3318 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 19:14:33.420856 3318 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ranger-kms
I20260811 19:14:33.420897 3318 mini_ranger_kms.cc:227] Using postgres at 127.3.61.148:60373
2026-08-11 19:14:33,835 [I] DB FLAVOR :POSTGRES
2026-08-11 19:14:33,835 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 19:14:33,835 [I] Checking connection
2026-08-11 19:14:33,835 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwEqXVd/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.61.148:60373/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 19:14:34,142 [I] connection success
2026-08-11 19:14:34,142 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 19:14:34,142 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 19:14:34,142 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwEqXVd/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.61.148:60373/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 19:14:34,407 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 19:14:34,407 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 19:14:34,407 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 19:14:34,408 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwEqXVd/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.61.148:60373/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 19:14:34,716 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 19:14:35.289208 3318 mini_ranger_kms.cc:310] Created kms service
I20260811 19:14:35.504009 3318 mini_ranger_kms.cc:326] Created kudu user
I20260811 19:14:35.552678 3318 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 19:14:35 dist-test-slave-t7g5 krb5kdc[23924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475672, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.3.61.148@KRBTEST.COM
I20260811 19:14:35.992780 3318 mini_ranger_kms.cc:384] Added ranger policy
I20260811 19:14:35.993124 3318 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ranger-kms:/tmp/dist-test-taskwEqXVd/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwEqXVd/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskwEqXVd/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 19:14:35.993166 3318 mini_ranger_kms.cc:240] Using host: 127.3.61.148
I20260811 19:14:36.026437 3318 mini_ranger_kms.cc:276] Ranger KMS PID: 24546
I20260811 19:14:36.026574 3318 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.3.61.148:34415
Aug 11, 2026 7:14:37 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 7:14:37 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 7:14:37 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 7:14:37 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 7:14:37 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 7:14:37 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 19:14:38 dist-test-slave-t7g5 krb5kdc[23924](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786475678, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.3.61.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:14:38.714536 3318 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.234s user 0.087s sys 0.369s
I20260811 19:14:38.714824 3318 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 19:14:38.714995 3318 mini_ranger_kms.cc:401] 127.3.61.148:34415/kms/v1/keys
Aug 11 19:14:39 dist-test-slave-t7g5 krb5kdc[23924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475678, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.3.61.148@KRBTEST.COM for HTTP/127.3.61.148@KRBTEST.COM
Aug 11 19:14:39 dist-test-slave-t7g5 krb5kdc[23924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475678, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.3.61.148@KRBTEST.COM for HTTP/127.3.61.148@KRBTEST.COM
Aug 11 19:14:42 dist-test-slave-t7g5 krb5kdc[23924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475682, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.3.61.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab.
Entry for principal kudu/127.3.61.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab.
I20260811 19:14:42.137835 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:42673
--webserver_interface=127.3.61.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:43131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.61.190:42673
--ranger_config_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.61.148:34415/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:14:42.508754 24622 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:14:42.509116 24622 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:14:42.509255 24622 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:14:42.516850 24622 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:14:42.517012 24622 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:14:42.517095 24622 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:14:42.517170 24622 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:14:42.517232 24622 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:14:42.527942 24622 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:43131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/master-0/wal
--ranger_kms_url=127.3.61.148:34415/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.3.61.190:42673
--ranger_config_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:42673
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.61.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:14:42.529381 24622 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:14:42.531113 24622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:14:42.545899 24628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:14:42.545908 24630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:14:42.546959 24632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:14:42.547565 24622 server_base.cc:1034] running on GCE node
I20260811 19:14:42.548645 24622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:14:42.550405 24622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:14:42.551657 24622 hybrid_clock.cc:648] HybridClock initialized: now 1786475682551609 us; error 45 us; skew 500 ppm
I20260811 19:14:42.552143 24622 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:14:42 dist-test-slave-t7g5 krb5kdc[23924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475682, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:14:42.555583 24622 init.cc:377] Logged in from keytab as kudu/127.3.61.190@KRBTEST.COM (short username kudu)
I20260811 19:14:42.557351 24622 webserver.cc:460] Webserver started at http://127.3.61.190:45903/ using document root <none> and password file <none>
I20260811 19:14:42.558107 24622 fs_manager.cc:359] Metadata directory not provided
I20260811 19:14:42.558230 24622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:14:42.558562 24622 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 19:14:42 dist-test-slave-t7g5 krb5kdc[23924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475682, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.190@KRBTEST.COM for HTTP/127.3.61.148@KRBTEST.COM
I20260811 19:14:42.622514 24622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/master-0/data/instance:
uuid: "062240d5b8e64e53818993a208e7cd8a"
format_stamp: "Formatted at 2026-08-11 19:14:42 on dist-test-slave-t7g5"
server_key: "9c29620e9469b850ffc031f485d46af0"
server_key_iv: "9a0a9430e5199afc283b7513f5325ed2"
server_key_version: "kuduclusterkey@0"
I20260811 19:14:42.623739 24622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/master-0/wal/instance:
uuid: "062240d5b8e64e53818993a208e7cd8a"
format_stamp: "Formatted at 2026-08-11 19:14:42 on dist-test-slave-t7g5"
server_key: "9c29620e9469b850ffc031f485d46af0"
server_key_iv: "9a0a9430e5199afc283b7513f5325ed2"
server_key_version: "kuduclusterkey@0"
I20260811 19:14:42.631536 24622 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.004s
I20260811 19:14:42.650203 24643 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:14:42.651676 24622 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260811 19:14:42.651865 24622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/master-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/master-0/wal
uuid: "062240d5b8e64e53818993a208e7cd8a"
format_stamp: "Formatted at 2026-08-11 19:14:42 on dist-test-slave-t7g5"
server_key: "9c29620e9469b850ffc031f485d46af0"
server_key_iv: "9a0a9430e5199afc283b7513f5325ed2"
server_key_version: "kuduclusterkey@0"
I20260811 19:14:42.652094 24622 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:14:42.707744 24622 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:14:42.712299 24622 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:14:42.712638 24622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:14:42.733018 24622 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.190:42673
I20260811 19:14:42.733054 24719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.190:42673 every 8 connection(s)
I20260811 19:14:42.734489 24622 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
I20260811 19:14:42.735780 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 24622
I20260811 19:14:42.736011 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/master-0/wal/instance
I20260811 19:14:42.736868 3318 external_mini_cluster.cc:1428] Setting key b6034824be43927ad5ea1bdeaffe40da
I20260811 19:14:42.739409 24720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 19:14:42 dist-test-slave-t7g5 krb5kdc[23924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475682, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:14:42.754218 24720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 062240d5b8e64e53818993a208e7cd8a: Bootstrap starting.
I20260811 19:14:42.757932 24720 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 062240d5b8e64e53818993a208e7cd8a: Neither blocks nor log segments found. Creating new log.
I20260811 19:14:42.759145 24720 log.cc:824] T 00000000000000000000000000000000 P 062240d5b8e64e53818993a208e7cd8a: Log is configured to *not* fsync() on all Append() calls
I20260811 19:14:42.762125 24723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:42.739060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59494 (local address 127.3.61.190:42673)
0811 19:14:42.739799 (+ 739us) server_negotiation.cc:206] Beginning negotiation
0811 19:14:42.739810 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 19:14:42.739852 (+ 42us) server_negotiation.cc:407] Connection header received
0811 19:14:42.741583 (+ 1731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:14:42.741605 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:14:42.742032 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:14:42.742383 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:14:42.744373 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:42.746434 (+ 2061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:42.748336 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:42.749248 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:42.749407 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:14:42.754342 (+ 4935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:14:42.754355 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:14:42.754378 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:14:42.754446 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:14:42.757712 (+ 3266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:14:42.758529 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:14:42.758537 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:14:42.758544 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:14:42.758697 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:14:42.759092 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:14:42.759097 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:14:42.759099 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:14:42.759769 (+ 670us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:14:42.760135 (+ 366us) server_negotiation.cc:1035] Waiting for connection context
0811 19:14:42.760599 (+ 464us) server_negotiation.cc:299] Negotiation successful
0811 19:14:42.761041 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":526,"thread_start_us":336,"threads_started":1}
I20260811 19:14:42.763311 24720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 062240d5b8e64e53818993a208e7cd8a: No bootstrap required, opened a new log
I20260811 19:14:42.771027 24720 raft_consensus.cc:348] T 00000000000000000000000000000000 P 062240d5b8e64e53818993a208e7cd8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "062240d5b8e64e53818993a208e7cd8a" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 42673 } }
I20260811 19:14:42.771634 24720 raft_consensus.cc:374] T 00000000000000000000000000000000 P 062240d5b8e64e53818993a208e7cd8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:14:42.771735 24720 raft_consensus.cc:729] T 00000000000000000000000000000000 P 062240d5b8e64e53818993a208e7cd8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 062240d5b8e64e53818993a208e7cd8a, State: Initialized, Role: FOLLOWER
I20260811 19:14:42.772334 24720 consensus_queue.cc:260] T 00000000000000000000000000000000 P 062240d5b8e64e53818993a208e7cd8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "062240d5b8e64e53818993a208e7cd8a" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 42673 } }
I20260811 19:14:42.772570 24720 raft_consensus.cc:388] T 00000000000000000000000000000000 P 062240d5b8e64e53818993a208e7cd8a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:14:42.772702 24720 raft_consensus.cc:482] T 00000000000000000000000000000000 P 062240d5b8e64e53818993a208e7cd8a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:14:42.772883 24720 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 062240d5b8e64e53818993a208e7cd8a [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:14:42.775282 24720 raft_consensus.cc:504] T 00000000000000000000000000000000 P 062240d5b8e64e53818993a208e7cd8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "062240d5b8e64e53818993a208e7cd8a" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 42673 } }
I20260811 19:14:42.775828 24720 leader_election.cc:302] T 00000000000000000000000000000000 P 062240d5b8e64e53818993a208e7cd8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 062240d5b8e64e53818993a208e7cd8a; no voters:
I20260811 19:14:42.776454 24720 leader_election.cc:288] T 00000000000000000000000000000000 P 062240d5b8e64e53818993a208e7cd8a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:14:42.776655 24725 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 062240d5b8e64e53818993a208e7cd8a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:14:42.777745 24725 raft_consensus.cc:686] T 00000000000000000000000000000000 P 062240d5b8e64e53818993a208e7cd8a [term 1 LEADER]: Becoming Leader. State: Replica: 062240d5b8e64e53818993a208e7cd8a, State: Running, Role: LEADER
I20260811 19:14:42.778379 24725 consensus_queue.cc:237] T 00000000000000000000000000000000 P 062240d5b8e64e53818993a208e7cd8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "062240d5b8e64e53818993a208e7cd8a" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 42673 } }
I20260811 19:14:42.778712 24720 sys_catalog.cc:564] T 00000000000000000000000000000000 P 062240d5b8e64e53818993a208e7cd8a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:14:42.782253 24727 sys_catalog.cc:455] T 00000000000000000000000000000000 P 062240d5b8e64e53818993a208e7cd8a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 062240d5b8e64e53818993a208e7cd8a. Latest consensus state: current_term: 1 leader_uuid: "062240d5b8e64e53818993a208e7cd8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "062240d5b8e64e53818993a208e7cd8a" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 42673 } } }
I20260811 19:14:42.782699 24727 sys_catalog.cc:458] T 00000000000000000000000000000000 P 062240d5b8e64e53818993a208e7cd8a [sys.catalog]: This master's current role is: LEADER
I20260811 19:14:42.782665 24726 sys_catalog.cc:455] T 00000000000000000000000000000000 P 062240d5b8e64e53818993a208e7cd8a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "062240d5b8e64e53818993a208e7cd8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "062240d5b8e64e53818993a208e7cd8a" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 42673 } } }
I20260811 19:14:42.783015 24726 sys_catalog.cc:458] T 00000000000000000000000000000000 P 062240d5b8e64e53818993a208e7cd8a [sys.catalog]: This master's current role is: LEADER
I20260811 19:14:42.784361 24736 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:14:42.790057 24736 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:14:42.795222 24720 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 19:14:42.808755 24736 catalog_manager.cc:1347] Generated new cluster ID: 6afe74250aa34f4f94b03c78616a0db9
I20260811 19:14:42.809087 24736 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:14:42.824489 24736 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:14:42.826231 24736 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:14:42.835893 24736 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 062240d5b8e64e53818993a208e7cd8a: Generated new TSK 0
I20260811 19:14:42.836807 24736 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.61.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab.
Entry for principal kudu/127.3.61.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab.
I20260811 19:14:43.759074 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.61.148:34415/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.129:0
--local_ip_for_outbound_sockets=127.3.61.129
--webserver_interface=127.3.61.129
--webserver_port=0
--tserver_master_addrs=127.3.61.190:42673
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab
--principal=kudu/127.3.61.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:43131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 19:14:44 dist-test-slave-t7g5 krb5kdc[23924](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786475684, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 19:14:44.250386 24794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:14:44.250854 24794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:14:44.251075 24794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:14:44.263465 24794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:14:44.263697 24794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:14:44.263913 24794 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.129
I20260811 19:14:44.280579 24794 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:43131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.3.61.148:34415/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab
--principal=kudu/127.3.61.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.61.129
--webserver_port=0
--tserver_master_addrs=127.3.61.190:42673
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.129
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:14:44.282724 24794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:14:44.284713 24794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:14:44.303141 24809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:14:44.303155 24813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:14:44.306052 24815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:14:44.313536 24794 server_base.cc:1034] running on GCE node
I20260811 19:14:44.314657 24794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:14:44.316356 24794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:14:44.317683 24794 hybrid_clock.cc:648] HybridClock initialized: now 1786475684317562 us; error 89 us; skew 500 ppm
I20260811 19:14:44.318161 24794 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:14:44 dist-test-slave-t7g5 krb5kdc[23924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475684, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:14:44.323805 24794 init.cc:377] Logged in from keytab as kudu/127.3.61.129@KRBTEST.COM (short username kudu)
I20260811 19:14:44.340355 24794 webserver.cc:460] Webserver started at http://127.3.61.129:34315/ using document root <none> and password file <none>
I20260811 19:14:44.341606 24794 fs_manager.cc:359] Metadata directory not provided
I20260811 19:14:44.341914 24794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:14:44.342404 24794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 19:14:44 dist-test-slave-t7g5 krb5kdc[23924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475684, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.129@KRBTEST.COM for HTTP/127.3.61.148@KRBTEST.COM
I20260811 19:14:44.370632 24794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-0/data/instance:
uuid: "e5ebae3dbafb413c82ec94f8cf6bd1fb"
format_stamp: "Formatted at 2026-08-11 19:14:44 on dist-test-slave-t7g5"
server_key: "929ad8be275bb68973bf727762479cfa"
server_key_iv: "5b375a3642317846391496f6c516ee4f"
server_key_version: "kuduclusterkey@0"
I20260811 19:14:44.373585 24794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-0/wal/instance:
uuid: "e5ebae3dbafb413c82ec94f8cf6bd1fb"
format_stamp: "Formatted at 2026-08-11 19:14:44 on dist-test-slave-t7g5"
server_key: "929ad8be275bb68973bf727762479cfa"
server_key_iv: "5b375a3642317846391496f6c516ee4f"
server_key_version: "kuduclusterkey@0"
I20260811 19:14:44.384845 24794 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.006s sys 0.004s
I20260811 19:14:44.422943 24823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:14:44.425371 24794 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.003s sys 0.007s
I20260811 19:14:44.425757 24794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-0/wal
uuid: "e5ebae3dbafb413c82ec94f8cf6bd1fb"
format_stamp: "Formatted at 2026-08-11 19:14:44 on dist-test-slave-t7g5"
server_key: "929ad8be275bb68973bf727762479cfa"
server_key_iv: "5b375a3642317846391496f6c516ee4f"
server_key_version: "kuduclusterkey@0"
I20260811 19:14:44.426281 24794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:14:44.478988 24794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:14:44.484916 24794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:14:44.485364 24794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:14:44.492993 24794 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:14:44.502435 24794 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:14:44.502655 24794 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260811 19:14:44.502885 24794 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:14:44.503007 24794 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 19:14:44 dist-test-slave-t7g5 krb5kdc[23924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475684, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.190@KRBTEST.COM for HTTP/127.3.61.148@KRBTEST.COM
Aug 11 19:14:45 dist-test-slave-t7g5 krb5kdc[23924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475684, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.190@KRBTEST.COM for HTTP/127.3.61.148@KRBTEST.COM
I20260811 19:14:45.073390 24794 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.129:38499
I20260811 19:14:45.074313 24998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.129:38499 every 8 connection(s)
I20260811 19:14:45.075323 24794 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
I20260811 19:14:45.076870 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 24794
I20260811 19:14:45.077092 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-0/wal/instance
I20260811 19:14:45.077989 3318 external_mini_cluster.cc:1428] Setting key b8b0f2940d719ca35995585d486db6d0
Aug 11 19:14:45 dist-test-slave-t7g5 krb5kdc[23924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475684, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.129@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:14:45.135054 25001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:45.090014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:33723 (local address 127.3.61.190:42673)
0811 19:14:45.106867 (+ 16853us) server_negotiation.cc:206] Beginning negotiation
0811 19:14:45.106874 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:14:45.106907 (+ 33us) server_negotiation.cc:407] Connection header received
0811 19:14:45.111503 (+ 4596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:14:45.111511 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:14:45.111620 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:14:45.111732 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:14:45.114921 (+ 3189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:45.117529 (+ 2608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:45.120476 (+ 2947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:45.121164 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:45.121279 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:14:45.126780 (+ 5501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:14:45.126789 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:14:45.126794 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:14:45.126855 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:14:45.130562 (+ 3707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:14:45.131759 (+ 1197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:14:45.131767 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:14:45.131771 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:14:45.131972 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:14:45.132635 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:14:45.132641 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:14:45.132644 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:14:45.133051 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:14:45.133195 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 19:14:45.134360 (+ 1165us) server_negotiation.cc:299] Negotiation successful
0811 19:14:45.134633 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16688,"thread_start_us":16424,"threads_started":1}
I20260811 19:14:45.137552 25002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:45.089375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:42673 (local address 127.3.61.129:33723)
0811 19:14:45.103281 (+ 13906us) negotiation.cc:107] Waiting for socket to connect
0811 19:14:45.103515 (+ 234us) client_negotiation.cc:174] Beginning negotiation
0811 19:14:45.110889 (+ 7374us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:14:45.112900 (+ 2011us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:14:45.112915 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:14:45.113694 (+ 779us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:14:45.114643 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:14:45.114674 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:14:45.117830 (+ 3156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:14:45.117837 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:14:45.120233 (+ 2396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:14:45.120257 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:14:45.121757 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:14:45.121766 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:14:45.122147 (+ 381us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:14:45.123078 (+ 931us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:14:45.123111 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:14:45.126237 (+ 3126us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:14:45.130823 (+ 4586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:14:45.130834 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:14:45.130856 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:14:45.131555 (+ 699us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:14:45.132140 (+ 585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:14:45.132146 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:14:45.132150 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:14:45.132466 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:14:45.133302 (+ 836us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:14:45.133311 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:14:45.133861 (+ 550us) client_negotiation.cc:769] Sending connection context
0811 19:14:45.135314 (+ 1453us) client_negotiation.cc:240] Negotiation successful
0811 19:14:45.135971 (+ 657us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13521,"thread_start_us":13259,"threads_started":1}
WARNING: no policy specified for kudu/127.3.61.130@KRBTEST.COM; defaulting to no policy
I20260811 19:14:45.139691 24999 heartbeater.cc:347] Connected to a master server at 127.3.61.190:42673
I20260811 19:14:45.140064 24999 heartbeater.cc:464] Registering TS with master...
I20260811 19:14:45.141000 24999 heartbeater.cc:511] Master 127.3.61.190:42673 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.130@KRBTEST.COM" created.
I20260811 19:14:45.144445 24669 ts_manager.cc:194] Registered new tserver with Master: e5ebae3dbafb413c82ec94f8cf6bd1fb (127.3.61.129:38499)
I20260811 19:14:45.146672 24669 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.129@KRBTEST.COM'} at 127.3.61.129:33723
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab.
Entry for principal kudu/127.3.61.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab.
W20260811 19:14:45.200754 3318 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.3.61.130: real 0.120s user 0.003s sys 0.025s
I20260811 19:14:45.201795 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.61.148:34415/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.130:0
--local_ip_for_outbound_sockets=127.3.61.130
--webserver_interface=127.3.61.130
--webserver_port=0
--tserver_master_addrs=127.3.61.190:42673
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab
--principal=kudu/127.3.61.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:43131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:14:45.609289 25007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:14:45.609630 25007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:14:45.609786 25007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:14:45.617962 25007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:14:45.618099 25007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:14:45.618214 25007 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.130
I20260811 19:14:45.628463 25007 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:43131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.3.61.148:34415/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab
--principal=kudu/127.3.61.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.61.130
--webserver_port=0
--tserver_master_addrs=127.3.61.190:42673
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.130
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:14:45.629863 25007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:14:45.631234 25007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:14:45.646060 25020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:14:45.646140 25021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:14:45.646209 25023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:14:45.647315 25007 server_base.cc:1034] running on GCE node
I20260811 19:14:45.648013 25007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:14:45.649070 25007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:14:45.650319 25007 hybrid_clock.cc:648] HybridClock initialized: now 1786475685650257 us; error 35 us; skew 500 ppm
I20260811 19:14:45.650677 25007 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:14:45 dist-test-slave-t7g5 krb5kdc[23924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475685, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:14:45.654712 25007 init.cc:377] Logged in from keytab as kudu/127.3.61.130@KRBTEST.COM (short username kudu)
I20260811 19:14:45.656639 25007 webserver.cc:460] Webserver started at http://127.3.61.130:42545/ using document root <none> and password file <none>
I20260811 19:14:45.657437 25007 fs_manager.cc:359] Metadata directory not provided
I20260811 19:14:45.657552 25007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:14:45.657912 25007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 19:14:45 dist-test-slave-t7g5 krb5kdc[23924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475685, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.130@KRBTEST.COM for HTTP/127.3.61.148@KRBTEST.COM
I20260811 19:14:45.675429 25007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-1/data/instance:
uuid: "ea56db5fc5d343fbaedbeae56b958918"
format_stamp: "Formatted at 2026-08-11 19:14:45 on dist-test-slave-t7g5"
server_key: "a6dd541d3bf3c8840fc89b39254a819e"
server_key_iv: "09905aeca186f5420d1c8340a5447993"
server_key_version: "kuduclusterkey@0"
I20260811 19:14:45.676939 25007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-1/wal/instance:
uuid: "ea56db5fc5d343fbaedbeae56b958918"
format_stamp: "Formatted at 2026-08-11 19:14:45 on dist-test-slave-t7g5"
server_key: "a6dd541d3bf3c8840fc89b39254a819e"
server_key_iv: "09905aeca186f5420d1c8340a5447993"
server_key_version: "kuduclusterkey@0"
I20260811 19:14:45.685261 25007 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.010s sys 0.000s
I20260811 19:14:45.703653 25036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:14:45.705349 25007 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.002s
I20260811 19:14:45.705541 25007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-1/wal
uuid: "ea56db5fc5d343fbaedbeae56b958918"
format_stamp: "Formatted at 2026-08-11 19:14:45 on dist-test-slave-t7g5"
server_key: "a6dd541d3bf3c8840fc89b39254a819e"
server_key_iv: "09905aeca186f5420d1c8340a5447993"
server_key_version: "kuduclusterkey@0"
I20260811 19:14:45.705837 25007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:14:45.750404 25007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:14:45.756328 25007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:14:45.756776 25007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:14:45.758015 25007 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:14:45.759629 25007 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:14:45.759711 25007 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:14:45.759837 25007 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:14:45.759887 25007 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:14:45.799129 25007 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.130:36265
I20260811 19:14:45.799134 25205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.130:36265 every 8 connection(s)
I20260811 19:14:45.800598 25007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
I20260811 19:14:45.804837 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 25007
I20260811 19:14:45.805025 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-1/wal/instance
I20260811 19:14:45.805493 3318 external_mini_cluster.cc:1428] Setting key 8cf77e3711d9e2ae25e2b1130f60abb4
Aug 11 19:14:45 dist-test-slave-t7g5 krb5kdc[23924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475685, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.130@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:14:45.823760 25208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:45.804158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:55391 (local address 127.3.61.190:42673)
0811 19:14:45.804783 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0811 19:14:45.804791 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:14:45.806653 (+ 1862us) server_negotiation.cc:407] Connection header received
0811 19:14:45.808337 (+ 1684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:14:45.808343 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:14:45.808433 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:14:45.808532 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:14:45.810176 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:45.811139 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:45.812742 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:45.813156 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:45.813270 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:14:45.817232 (+ 3962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:14:45.817238 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:14:45.817242 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:14:45.817283 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:14:45.819760 (+ 2477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:14:45.820665 (+ 905us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:14:45.820670 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:14:45.820673 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:14:45.820824 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:14:45.821367 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:14:45.821371 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:14:45.821373 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:14:45.821720 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:14:45.821841 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 19:14:45.822951 (+ 1110us) server_negotiation.cc:299] Negotiation successful
0811 19:14:45.823245 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":315,"threads_started":1}
I20260811 19:14:45.825335 25209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:45.804780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:42673 (local address 127.3.61.130:55391)
0811 19:14:45.806115 (+ 1335us) negotiation.cc:107] Waiting for socket to connect
0811 19:14:45.806169 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 19:14:45.808065 (+ 1896us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:14:45.808801 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:14:45.808811 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:14:45.809273 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:14:45.810000 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:14:45.810023 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:14:45.811298 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:14:45.811310 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:14:45.812595 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:14:45.812609 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:14:45.813403 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:14:45.813411 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:14:45.813647 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:14:45.814491 (+ 844us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:14:45.814524 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:14:45.816986 (+ 2462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:14:45.819923 (+ 2937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:14:45.819933 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:14:45.819948 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:14:45.820502 (+ 554us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:14:45.820948 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:14:45.820952 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:14:45.820956 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:14:45.821235 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:14:45.821844 (+ 609us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:14:45.821857 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:14:45.822295 (+ 438us) client_negotiation.cc:769] Sending connection context
0811 19:14:45.822614 (+ 319us) client_negotiation.cc:240] Negotiation successful
0811 19:14:45.823522 (+ 908us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1005,"thread_start_us":617,"threads_started":1}
I20260811 19:14:45.827438 25206 heartbeater.cc:347] Connected to a master server at 127.3.61.190:42673
I20260811 19:14:45.827808 25206 heartbeater.cc:464] Registering TS with master...
I20260811 19:14:45.828758 25206 heartbeater.cc:511] Master 127.3.61.190:42673 requested a full tablet report, sending...
I20260811 19:14:45.830410 24669 ts_manager.cc:194] Registered new tserver with Master: ea56db5fc5d343fbaedbeae56b958918 (127.3.61.130:36265)
I20260811 19:14:45.831228 24669 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.130@KRBTEST.COM'} at 127.3.61.130:55391
WARNING: no policy specified for kudu/127.3.61.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab.
Entry for principal kudu/127.3.61.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab.
I20260811 19:14:45.876755 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.61.148:34415/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.131:0
--local_ip_for_outbound_sockets=127.3.61.131
--webserver_interface=127.3.61.131
--webserver_port=0
--tserver_master_addrs=127.3.61.190:42673
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab
--principal=kudu/127.3.61.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:43131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:14:46.150908 24999 heartbeater.cc:503] Master 127.3.61.190:42673 was elected leader, sending a full tablet report...
W20260811 19:14:46.258414 25213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:14:46.258735 25213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:14:46.258852 25213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:14:46.267867 25213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:14:46.268075 25213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:14:46.268347 25213 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.131
I20260811 19:14:46.282245 25213 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:43131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.3.61.148:34415/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab
--principal=kudu/127.3.61.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.61.131
--webserver_port=0
--tserver_master_addrs=127.3.61.190:42673
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.131
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:14:46.283792 25213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:14:46.285244 25213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:14:46.299623 25220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:14:46.299679 25221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:14:46.299970 25225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:14:46.300992 25213 server_base.cc:1034] running on GCE node
I20260811 19:14:46.301841 25213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:14:46.303222 25213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:14:46.304457 25213 hybrid_clock.cc:648] HybridClock initialized: now 1786475686304385 us; error 66 us; skew 500 ppm
I20260811 19:14:46.304898 25213 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:14:46 dist-test-slave-t7g5 krb5kdc[23924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475686, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:14:46.309593 25213 init.cc:377] Logged in from keytab as kudu/127.3.61.131@KRBTEST.COM (short username kudu)
I20260811 19:14:46.311753 25213 webserver.cc:460] Webserver started at http://127.3.61.131:45315/ using document root <none> and password file <none>
I20260811 19:14:46.312559 25213 fs_manager.cc:359] Metadata directory not provided
I20260811 19:14:46.312693 25213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:14:46.313041 25213 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 19:14:46 dist-test-slave-t7g5 krb5kdc[23924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475686, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.131@KRBTEST.COM for HTTP/127.3.61.148@KRBTEST.COM
I20260811 19:14:46.333818 25213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-2/data/instance:
uuid: "c136675655204b019dd2894c3b373522"
format_stamp: "Formatted at 2026-08-11 19:14:46 on dist-test-slave-t7g5"
server_key: "804a6705690df33b5f0dcb24b74789b8"
server_key_iv: "7b0783c792295d850dd44c1152b4a2f3"
server_key_version: "kuduclusterkey@0"
I20260811 19:14:46.335103 25213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-2/wal/instance:
uuid: "c136675655204b019dd2894c3b373522"
format_stamp: "Formatted at 2026-08-11 19:14:46 on dist-test-slave-t7g5"
server_key: "804a6705690df33b5f0dcb24b74789b8"
server_key_iv: "7b0783c792295d850dd44c1152b4a2f3"
server_key_version: "kuduclusterkey@0"
I20260811 19:14:46.343693 25213 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.004s sys 0.004s
I20260811 19:14:46.364044 25237 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:14:46.366055 25213 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260811 19:14:46.366307 25213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-2/wal
uuid: "c136675655204b019dd2894c3b373522"
format_stamp: "Formatted at 2026-08-11 19:14:46 on dist-test-slave-t7g5"
server_key: "804a6705690df33b5f0dcb24b74789b8"
server_key_iv: "7b0783c792295d850dd44c1152b4a2f3"
server_key_version: "kuduclusterkey@0"
I20260811 19:14:46.366603 25213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:14:46.391848 25213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:14:46.397810 25213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:14:46.398137 25213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:14:46.399287 25213 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:14:46.400899 25213 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:14:46.400979 25213 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:14:46.401104 25213 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:14:46.401157 25213 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:14:46.439109 25213 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.131:40539
I20260811 19:14:46.439133 25413 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.131:40539 every 8 connection(s)
I20260811 19:14:46.440502 25213 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
I20260811 19:14:46.442394 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 25213
I20260811 19:14:46.442571 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-2/wal/instance
I20260811 19:14:46.443145 3318 external_mini_cluster.cc:1428] Setting key aa604d2f4327d9117527e10e9d6da392
Aug 11 19:14:46 dist-test-slave-t7g5 krb5kdc[23924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475686, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.131@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:14:46.466934 25417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:46.444892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:48367 (local address 127.3.61.190:42673)
0811 19:14:46.450250 (+ 5358us) server_negotiation.cc:206] Beginning negotiation
0811 19:14:46.450255 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:14:46.450274 (+ 19us) server_negotiation.cc:407] Connection header received
0811 19:14:46.450327 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:14:46.450331 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:14:46.450407 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:14:46.450532 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:14:46.452216 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:46.452978 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:46.454623 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:46.455080 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:46.455221 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:14:46.460460 (+ 5239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:14:46.460466 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:14:46.460469 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:14:46.460504 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:14:46.463409 (+ 2905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:14:46.464317 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:14:46.464322 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:14:46.464324 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:14:46.464472 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:14:46.464958 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:14:46.464962 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:14:46.464964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:14:46.465255 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:14:46.465409 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 19:14:46.466258 (+ 849us) server_negotiation.cc:299] Negotiation successful
0811 19:14:46.466549 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5245,"thread_start_us":5148,"threads_started":1}
I20260811 19:14:46.468394 25416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:46.444204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:42673 (local address 127.3.61.131:48367)
0811 19:14:46.445324 (+ 1120us) negotiation.cc:107] Waiting for socket to connect
0811 19:14:46.445382 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 19:14:46.447385 (+ 2003us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:14:46.450863 (+ 3478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:14:46.450873 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:14:46.451374 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:14:46.452024 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:14:46.452044 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:14:46.453132 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:14:46.453136 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:14:46.454474 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:14:46.454489 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:14:46.455340 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:14:46.455349 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:14:46.455632 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:14:46.456552 (+ 920us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:14:46.456602 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:14:46.460217 (+ 3615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:14:46.463596 (+ 3379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:14:46.463609 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:14:46.463627 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:14:46.464176 (+ 549us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:14:46.464587 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:14:46.464592 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:14:46.464596 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:14:46.464826 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:14:46.465401 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:14:46.465410 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:14:46.465835 (+ 425us) client_negotiation.cc:769] Sending connection context
0811 19:14:46.466115 (+ 280us) client_negotiation.cc:240] Negotiation successful
0811 19:14:46.466724 (+ 609us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":770,"thread_start_us":731,"threads_started":1}
I20260811 19:14:46.470878 25414 heartbeater.cc:347] Connected to a master server at 127.3.61.190:42673
I20260811 19:14:46.471297 25414 heartbeater.cc:464] Registering TS with master...
I20260811 19:14:46.472280 25414 heartbeater.cc:511] Master 127.3.61.190:42673 requested a full tablet report, sending...
I20260811 19:14:46.474112 24669 ts_manager.cc:194] Registered new tserver with Master: c136675655204b019dd2894c3b373522 (127.3.61.131:40539)
I20260811 19:14:46.474953 24669 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.131@KRBTEST.COM'} at 127.3.61.131:48367
I20260811 19:14:46.486387 3318 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 19:14:46.488245 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 24794
Aug 11 19:14:46 dist-test-slave-t7g5 krb5kdc[23924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475682, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.129@KRBTEST.COM
I20260811 19:14:46.510838 25420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:46.489377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59426 (local address 127.3.61.129:38499)
0811 19:14:46.490205 (+ 828us) server_negotiation.cc:206] Beginning negotiation
0811 19:14:46.490214 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:14:46.490600 (+ 386us) server_negotiation.cc:407] Connection header received
0811 19:14:46.490747 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:14:46.490756 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:14:46.491061 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:14:46.491233 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:14:46.492706 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:46.494367 (+ 1661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:46.496201 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:46.497121 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:46.497266 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:14:46.503137 (+ 5871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:14:46.503161 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:14:46.503185 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:14:46.503247 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:14:46.507155 (+ 3908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:14:46.507995 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:14:46.508005 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:14:46.508009 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:14:46.508195 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:14:46.508570 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:14:46.508575 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:14:46.508578 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:14:46.509051 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:14:46.509403 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0811 19:14:46.510081 (+ 678us) server_negotiation.cc:299] Negotiation successful
0811 19:14:46.510392 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":642,"thread_start_us":458,"threads_started":1}
I20260811 19:14:46.512444 24992 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:14:46.599664 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 24794
I20260811 19:14:46.627929 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 25007
Aug 11 19:14:46 dist-test-slave-t7g5 krb5kdc[23924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475682, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.130@KRBTEST.COM
I20260811 19:14:46.647747 25422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:46.628941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43032 (local address 127.3.61.130:36265)
0811 19:14:46.629560 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0811 19:14:46.629569 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:14:46.629598 (+ 29us) server_negotiation.cc:407] Connection header received
0811 19:14:46.629698 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:14:46.629707 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:14:46.629978 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:14:46.630139 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:14:46.631628 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:46.632692 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:46.634066 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:46.634663 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:46.634802 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:14:46.639940 (+ 5138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:14:46.639955 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:14:46.639971 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:14:46.640013 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:14:46.644409 (+ 4396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:14:46.645391 (+ 982us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:14:46.645402 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:14:46.645406 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:14:46.645559 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:14:46.645959 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:14:46.645964 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:14:46.645968 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:14:46.646358 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:14:46.646684 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0811 19:14:46.647051 (+ 367us) server_negotiation.cc:299] Negotiation successful
0811 19:14:46.647338 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":463,"thread_start_us":330,"threads_started":1}
I20260811 19:14:46.649236 25201 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:14:46.737948 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 25007
I20260811 19:14:46.762045 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 25213
Aug 11 19:14:46 dist-test-slave-t7g5 krb5kdc[23924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475682, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.131@KRBTEST.COM
I20260811 19:14:46.783919 25425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:46.763468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38712 (local address 127.3.61.131:40539)
0811 19:14:46.764272 (+ 804us) server_negotiation.cc:206] Beginning negotiation
0811 19:14:46.764282 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 19:14:46.764326 (+ 44us) server_negotiation.cc:407] Connection header received
0811 19:14:46.764415 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:14:46.764424 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:14:46.764815 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:14:46.765070 (+ 255us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:14:46.766561 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:46.767714 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:46.769808 (+ 2094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:46.770429 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:46.770579 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:14:46.775563 (+ 4984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:14:46.775581 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:14:46.775603 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:14:46.775659 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:14:46.779966 (+ 4307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:14:46.780948 (+ 982us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:14:46.780961 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:14:46.780965 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:14:46.781244 (+ 279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:14:46.781687 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:14:46.781692 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:14:46.781695 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:14:46.782292 (+ 597us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:14:46.782711 (+ 419us) server_negotiation.cc:1035] Waiting for connection context
0811 19:14:46.782994 (+ 283us) server_negotiation.cc:299] Negotiation successful
0811 19:14:46.783371 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":305,"threads_started":1}
I20260811 19:14:46.785450 25409 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:14:46.872572 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 25213
I20260811 19:14:46.898108 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 24622
I20260811 19:14:46.899847 24711 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:14:47.014655 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 24622
I20260811 19:14:47.047688 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:42673
--webserver_interface=127.3.61.190
--webserver_port=45903
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:43131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.61.190:42673
--ranger_config_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.61.148:34415/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.3.61.148:34415/kms with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:14:47.421990 25428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:14:47.422302 25428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:14:47.422410 25428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:14:47.430337 25428 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:14:47.430472 25428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:14:47.430519 25428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:14:47.430576 25428 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:14:47.430624 25428 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:14:47.441421 25428 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:43131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.3.61.148:34415/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.3.61.190:42673
--ranger_config_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:42673
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.61.190
--webserver_port=45903
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:14:47.442802 25428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:14:47.444120 25428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:14:47.456507 25439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:14:47.456481 25440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:14:47.456530 25444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:14:47.457103 25428 server_base.cc:1034] running on GCE node
I20260811 19:14:47.458042 25428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:14:47.459533 25428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:14:47.460770 25428 hybrid_clock.cc:648] HybridClock initialized: now 1786475687460728 us; error 41 us; skew 500 ppm
I20260811 19:14:47.461163 25428 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:14:47 dist-test-slave-t7g5 krb5kdc[23924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475687, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:14:47.464840 25428 init.cc:377] Logged in from keytab as kudu/127.3.61.190@KRBTEST.COM (short username kudu)
I20260811 19:14:47.466720 25428 webserver.cc:460] Webserver started at http://127.3.61.190:45903/ using document root <none> and password file <none>
I20260811 19:14:47.467442 25428 fs_manager.cc:359] Metadata directory not provided
I20260811 19:14:47.467576 25428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 19:14:47 dist-test-slave-t7g5 krb5kdc[23924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475687, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.190@KRBTEST.COM for HTTP/127.3.61.148@KRBTEST.COM
I20260811 19:14:48.002683 25428 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 19:14:48.006387 25455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:14:48.008098 25428 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260811 19:14:48.008303 25428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/master-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/master-0/wal
uuid: "062240d5b8e64e53818993a208e7cd8a"
format_stamp: "Formatted at 2026-08-11 19:14:42 on dist-test-slave-t7g5"
server_key: "9c29620e9469b850ffc031f485d46af0"
server_key_iv: "9a0a9430e5199afc283b7513f5325ed2"
server_key_version: "kuduclusterkey@0"
I20260811 19:14:48.009353 25428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:14:48.028499 25428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:14:48.035173 25428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:14:48.035571 25428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:14:48.058295 25428 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.190:42673
I20260811 19:14:48.058553 25531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.190:42673 every 8 connection(s)
I20260811 19:14:48.060156 25428 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
I20260811 19:14:48.067113 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 25428
I20260811 19:14:48.068243 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.61.148:34415/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.129:38499
--local_ip_for_outbound_sockets=127.3.61.129
--tserver_master_addrs=127.3.61.190:42673
--webserver_port=34315
--webserver_interface=127.3.61.129
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab
--principal=kudu/127.3.61.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:43131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.3.61.148:34415/kms with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:14:48.068385 25535 sys_catalog.cc:263] Verifying existing consensus state
I20260811 19:14:48.074716 25535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 062240d5b8e64e53818993a208e7cd8a: Bootstrap starting.
I20260811 19:14:48.097808 25535 log.cc:824] T 00000000000000000000000000000000 P 062240d5b8e64e53818993a208e7cd8a: Log is configured to *not* fsync() on all Append() calls
I20260811 19:14:48.108353 25535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 062240d5b8e64e53818993a208e7cd8a: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 19:14:48.108922 25535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 062240d5b8e64e53818993a208e7cd8a: Bootstrap complete.
I20260811 19:14:48.119073 25535 raft_consensus.cc:348] T 00000000000000000000000000000000 P 062240d5b8e64e53818993a208e7cd8a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "062240d5b8e64e53818993a208e7cd8a" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 42673 } }
I20260811 19:14:48.120014 25535 raft_consensus.cc:729] T 00000000000000000000000000000000 P 062240d5b8e64e53818993a208e7cd8a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 062240d5b8e64e53818993a208e7cd8a, State: Initialized, Role: FOLLOWER
I20260811 19:14:48.120656 25535 consensus_queue.cc:260] T 00000000000000000000000000000000 P 062240d5b8e64e53818993a208e7cd8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "062240d5b8e64e53818993a208e7cd8a" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 42673 } }
I20260811 19:14:48.120893 25535 raft_consensus.cc:388] T 00000000000000000000000000000000 P 062240d5b8e64e53818993a208e7cd8a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:14:48.121006 25535 raft_consensus.cc:482] T 00000000000000000000000000000000 P 062240d5b8e64e53818993a208e7cd8a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:14:48.121173 25535 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 062240d5b8e64e53818993a208e7cd8a [term 1 FOLLOWER]: Advancing to term 2
I20260811 19:14:48.125068 25535 raft_consensus.cc:504] T 00000000000000000000000000000000 P 062240d5b8e64e53818993a208e7cd8a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "062240d5b8e64e53818993a208e7cd8a" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 42673 } }
I20260811 19:14:48.125576 25535 leader_election.cc:302] T 00000000000000000000000000000000 P 062240d5b8e64e53818993a208e7cd8a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 062240d5b8e64e53818993a208e7cd8a; no voters:
I20260811 19:14:48.126231 25535 leader_election.cc:288] T 00000000000000000000000000000000 P 062240d5b8e64e53818993a208e7cd8a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 19:14:48.126442 25540 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 062240d5b8e64e53818993a208e7cd8a [term 2 FOLLOWER]: Leader election won for term 2
I20260811 19:14:48.127513 25540 raft_consensus.cc:686] T 00000000000000000000000000000000 P 062240d5b8e64e53818993a208e7cd8a [term 2 LEADER]: Becoming Leader. State: Replica: 062240d5b8e64e53818993a208e7cd8a, State: Running, Role: LEADER
I20260811 19:14:48.128093 25540 consensus_queue.cc:237] T 00000000000000000000000000000000 P 062240d5b8e64e53818993a208e7cd8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "062240d5b8e64e53818993a208e7cd8a" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 42673 } }
I20260811 19:14:48.128495 25535 sys_catalog.cc:564] T 00000000000000000000000000000000 P 062240d5b8e64e53818993a208e7cd8a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:14:48.131131 25543 sys_catalog.cc:455] T 00000000000000000000000000000000 P 062240d5b8e64e53818993a208e7cd8a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 062240d5b8e64e53818993a208e7cd8a. Latest consensus state: current_term: 2 leader_uuid: "062240d5b8e64e53818993a208e7cd8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "062240d5b8e64e53818993a208e7cd8a" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 42673 } } }
I20260811 19:14:48.131110 25542 sys_catalog.cc:455] T 00000000000000000000000000000000 P 062240d5b8e64e53818993a208e7cd8a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "062240d5b8e64e53818993a208e7cd8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "062240d5b8e64e53818993a208e7cd8a" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 42673 } } }
I20260811 19:14:48.131716 25543 sys_catalog.cc:458] T 00000000000000000000000000000000 P 062240d5b8e64e53818993a208e7cd8a [sys.catalog]: This master's current role is: LEADER
I20260811 19:14:48.131805 25542 sys_catalog.cc:458] T 00000000000000000000000000000000 P 062240d5b8e64e53818993a208e7cd8a [sys.catalog]: This master's current role is: LEADER
I20260811 19:14:48.138259 25549 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:14:48.144872 25549 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:14:48.146466 25549 catalog_manager.cc:1259] Loaded cluster ID: 6afe74250aa34f4f94b03c78616a0db9
I20260811 19:14:48.146775 25549 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:14:48.147823 25535 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 19:14:48.152405 25549 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:14:48.154906 25549 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 062240d5b8e64e53818993a208e7cd8a: Loaded TSK: 0
I20260811 19:14:48.164232 25549 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 19:14:48.568384 25538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:14:48.568841 25538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:14:48.569022 25538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:14:48.580909 25538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:14:48.581079 25538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:14:48.581249 25538 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.129
I20260811 19:14:48.598160 25538 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:43131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.3.61.148:34415/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab
--principal=kudu/127.3.61.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.129:38499
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.61.129
--webserver_port=34315
--tserver_master_addrs=127.3.61.190:42673
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.129
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:14:48.600041 25538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:14:48.601600 25538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:14:48.631637 25592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:14:48.631747 25594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:14:48.639407 25598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:14:48.640409 25538 server_base.cc:1034] running on GCE node
I20260811 19:14:48.641170 25538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:14:48.642395 25538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:14:48.643662 25538 hybrid_clock.cc:648] HybridClock initialized: now 1786475688643591 us; error 52 us; skew 500 ppm
I20260811 19:14:48.644084 25538 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:14:48 dist-test-slave-t7g5 krb5kdc[23924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475688, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:14:48.652801 25538 init.cc:377] Logged in from keytab as kudu/127.3.61.129@KRBTEST.COM (short username kudu)
I20260811 19:14:48.658334 25538 webserver.cc:460] Webserver started at http://127.3.61.129:34315/ using document root <none> and password file <none>
I20260811 19:14:48.659246 25538 fs_manager.cc:359] Metadata directory not provided
I20260811 19:14:48.659422 25538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 19:14:48 dist-test-slave-t7g5 krb5kdc[23924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475688, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.129@KRBTEST.COM for HTTP/127.3.61.148@KRBTEST.COM
I20260811 19:14:48.693554 25538 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.005s sys 0.000s
I20260811 19:14:48.710335 25607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:14:48.712558 25538 fs_manager.cc:725] Time spent opening block manager: real 0.017s user 0.011s sys 0.005s
I20260811 19:14:48.712875 25538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-0/wal
uuid: "e5ebae3dbafb413c82ec94f8cf6bd1fb"
format_stamp: "Formatted at 2026-08-11 19:14:44 on dist-test-slave-t7g5"
server_key: "929ad8be275bb68973bf727762479cfa"
server_key_iv: "5b375a3642317846391496f6c516ee4f"
server_key_version: "kuduclusterkey@0"
I20260811 19:14:48.714435 25538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:14:48.744630 25538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:14:48.749967 25538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:14:48.750294 25538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:14:48.751645 25538 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:14:48.753325 25538 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:14:48.753423 25538 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:14:48.753537 25538 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:14:48.753607 25538 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:14:48.830649 25538 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.129:38499
I20260811 19:14:48.830685 25795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.129:38499 every 8 connection(s)
I20260811 19:14:48.832420 25538 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
I20260811 19:14:48.845017 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 25538
I20260811 19:14:48.846303 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.61.148:34415/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.130:36265
--local_ip_for_outbound_sockets=127.3.61.130
--tserver_master_addrs=127.3.61.190:42673
--webserver_port=42545
--webserver_interface=127.3.61.130
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab
--principal=kudu/127.3.61.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:43131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.3.61.148:34415/kms with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 19:14:48 dist-test-slave-t7g5 krb5kdc[23924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475688, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.129@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:14:48.902546 25799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:48.838207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:53159 (local address 127.3.61.190:42673)
0811 19:14:48.858216 (+ 20009us) server_negotiation.cc:206] Beginning negotiation
0811 19:14:48.858235 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 19:14:48.869369 (+ 11134us) server_negotiation.cc:407] Connection header received
0811 19:14:48.874668 (+ 5299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:14:48.874694 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:14:48.875359 (+ 665us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:14:48.875810 (+ 451us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:14:48.878732 (+ 2922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:48.880774 (+ 2042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:48.883540 (+ 2766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:48.884728 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:48.884946 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:14:48.890192 (+ 5246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:14:48.890207 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:14:48.890234 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:14:48.890302 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:14:48.894441 (+ 4139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:14:48.895650 (+ 1209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:14:48.895663 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:14:48.895672 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:14:48.895915 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:14:48.896496 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:14:48.896502 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:14:48.896506 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:14:48.897218 (+ 712us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:14:48.897433 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0811 19:14:48.900705 (+ 3272us) server_negotiation.cc:299] Negotiation successful
0811 19:14:48.901200 (+ 495us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19729,"thread_start_us":19507,"threads_started":1}
I20260811 19:14:48.903621 25798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:48.836799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:42673 (local address 127.3.61.129:53159)
0811 19:14:48.869075 (+ 32276us) negotiation.cc:107] Waiting for socket to connect
0811 19:14:48.869150 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0811 19:14:48.873070 (+ 3920us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:14:48.876810 (+ 3740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:14:48.876821 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:14:48.877467 (+ 646us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:14:48.878397 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:14:48.878435 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:14:48.881026 (+ 2591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:14:48.881032 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:14:48.883257 (+ 2225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:14:48.883308 (+ 51us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:14:48.885255 (+ 1947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:14:48.885266 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:14:48.885588 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:14:48.886511 (+ 923us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:14:48.886550 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:14:48.889737 (+ 3187us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:14:48.894697 (+ 4960us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:14:48.894709 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:14:48.894728 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:14:48.895423 (+ 695us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:14:48.896072 (+ 649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:14:48.896078 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:14:48.896082 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:14:48.896356 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:14:48.897523 (+ 1167us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:14:48.897532 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:14:48.898009 (+ 477us) client_negotiation.cc:769] Sending connection context
0811 19:14:48.898345 (+ 336us) client_negotiation.cc:240] Negotiation successful
0811 19:14:48.898975 (+ 630us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31827,"thread_start_us":32480,"threads_started":1}
I20260811 19:14:48.907547 25796 heartbeater.cc:347] Connected to a master server at 127.3.61.190:42673
I20260811 19:14:48.907873 25796 heartbeater.cc:464] Registering TS with master...
I20260811 19:14:48.908721 25796 heartbeater.cc:511] Master 127.3.61.190:42673 requested a full tablet report, sending...
W20260811 19:14:48.912642 25796 heartbeater.cc:644] Failed to heartbeat to 127.3.61.190:42673 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 19:14:48.912932 25796 heartbeater.cc:464] Registering TS with master...
I20260811 19:14:48.913313 25796 heartbeater.cc:511] Master 127.3.61.190:42673 requested a full tablet report, sending...
I20260811 19:14:48.914861 25796 heartbeater.cc:464] Registering TS with master...
I20260811 19:14:48.915194 25796 heartbeater.cc:511] Master 127.3.61.190:42673 requested a full tablet report, sending...
W20260811 19:14:48.916657 25796 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 19:14:49.298413 25800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:14:49.298846 25800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:14:49.299012 25800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:14:49.308097 25800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:14:49.308234 25800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:14:49.308386 25800 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.130
I20260811 19:14:49.319672 25800 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:43131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.3.61.148:34415/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab
--principal=kudu/127.3.61.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.130:36265
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.61.130
--webserver_port=42545
--tserver_master_addrs=127.3.61.190:42673
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.130
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:14:49.321167 25800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:14:49.322563 25800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:14:49.338764 25820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:14:49.338762 25818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:14:49.347962 25824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:14:49.348313 25800 server_base.cc:1034] running on GCE node
I20260811 19:14:49.349321 25800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:14:49.350836 25800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:14:49.352124 25800 hybrid_clock.cc:648] HybridClock initialized: now 1786475689352049 us; error 63 us; skew 500 ppm
I20260811 19:14:49.352576 25800 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:14:49 dist-test-slave-t7g5 krb5kdc[23924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475689, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:14:49.357465 25800 init.cc:377] Logged in from keytab as kudu/127.3.61.130@KRBTEST.COM (short username kudu)
I20260811 19:14:49.359942 25800 webserver.cc:460] Webserver started at http://127.3.61.130:42545/ using document root <none> and password file <none>
I20260811 19:14:49.361011 25800 fs_manager.cc:359] Metadata directory not provided
I20260811 19:14:49.361199 25800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 19:14:49 dist-test-slave-t7g5 krb5kdc[23924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475689, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.130@KRBTEST.COM for HTTP/127.3.61.148@KRBTEST.COM
I20260811 19:14:49.396510 25800 fs_manager.cc:709] Time spent opening directory manager: real 0.013s user 0.007s sys 0.004s
I20260811 19:14:49.411468 25836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:14:49.413906 25800 fs_manager.cc:725] Time spent opening block manager: real 0.015s user 0.011s sys 0.004s
I20260811 19:14:49.414179 25800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-1/wal
uuid: "ea56db5fc5d343fbaedbeae56b958918"
format_stamp: "Formatted at 2026-08-11 19:14:45 on dist-test-slave-t7g5"
server_key: "a6dd541d3bf3c8840fc89b39254a819e"
server_key_iv: "09905aeca186f5420d1c8340a5447993"
server_key_version: "kuduclusterkey@0"
I20260811 19:14:49.415787 25800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:14:49.476972 25800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:14:49.481235 25800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:14:49.481544 25800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:14:49.482767 25800 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:14:49.484334 25800 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:14:49.484437 25800 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:14:49.484557 25800 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:14:49.484637 25800 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 19:14:49 dist-test-slave-t7g5 krb5kdc[23924](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786475689, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:14:49.560324 25800 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.130:36265
I20260811 19:14:49.560379 25999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.130:36265 every 8 connection(s)
I20260811 19:14:49.562203 25800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
I20260811 19:14:49.568878 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 25800
I20260811 19:14:49.569732 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.61.148:34415/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.131:40539
--local_ip_for_outbound_sockets=127.3.61.131
--tserver_master_addrs=127.3.61.190:42673
--webserver_port=45315
--webserver_interface=127.3.61.131
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab
--principal=kudu/127.3.61.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:43131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.3.61.148:34415/kms with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 19:14:49 dist-test-slave-t7g5 krb5kdc[23924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475689, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.130@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:14:49.588068 26005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:49.565545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:42673 (local address 127.3.61.130:35107)
0811 19:14:49.566550 (+ 1005us) negotiation.cc:107] Waiting for socket to connect
0811 19:14:49.566601 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 19:14:49.568486 (+ 1885us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:14:49.569288 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:14:49.569297 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:14:49.570547 (+ 1250us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:14:49.571218 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:14:49.571239 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:14:49.572399 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:14:49.572403 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:14:49.573785 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:14:49.573799 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:14:49.574641 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:14:49.574649 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:14:49.574978 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:14:49.575678 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:14:49.575701 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:14:49.577796 (+ 2095us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:14:49.581582 (+ 3786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:14:49.581594 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:14:49.581618 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:14:49.582312 (+ 694us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:14:49.582890 (+ 578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:14:49.582898 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:14:49.582903 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:14:49.583354 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:14:49.584394 (+ 1040us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:14:49.584404 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:14:49.584949 (+ 545us) client_negotiation.cc:769] Sending connection context
0811 19:14:49.585410 (+ 461us) client_negotiation.cc:240] Negotiation successful
0811 19:14:49.586422 (+ 1012us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":714,"thread_start_us":390,"threads_started":1}
I20260811 19:14:49.589813 26004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:49.564995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:35107 (local address 127.3.61.190:42673)
0811 19:14:49.565532 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0811 19:14:49.565539 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:14:49.566753 (+ 1214us) server_negotiation.cc:407] Connection header received
0811 19:14:49.568799 (+ 2046us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:14:49.568805 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:14:49.568890 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:14:49.568989 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:14:49.571425 (+ 2436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:49.572225 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:49.573935 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:49.574382 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:49.574501 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:14:49.578048 (+ 3547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:14:49.578054 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:14:49.578058 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:14:49.578100 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:14:49.581275 (+ 3175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:14:49.582470 (+ 1195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:14:49.582475 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:14:49.582477 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:14:49.582672 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:14:49.583531 (+ 859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:14:49.583537 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:14:49.583540 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:14:49.584166 (+ 626us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:14:49.584383 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0811 19:14:49.585559 (+ 1176us) server_negotiation.cc:299] Negotiation successful
0811 19:14:49.589381 (+ 3822us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":264,"threads_started":1}
I20260811 19:14:49.591468 26001 heartbeater.cc:347] Connected to a master server at 127.3.61.190:42673
I20260811 19:14:49.592036 26001 heartbeater.cc:464] Registering TS with master...
I20260811 19:14:49.593386 26001 heartbeater.cc:511] Master 127.3.61.190:42673 requested a full tablet report, sending...
I20260811 19:14:49.596391 25485 ts_manager.cc:194] Registered new tserver with Master: ea56db5fc5d343fbaedbeae56b958918 (127.3.61.130:36265)
I20260811 19:14:49.598850 25485 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.130@KRBTEST.COM'} at 127.3.61.130:35107
I20260811 19:14:49.916857 25796 heartbeater.cc:464] Registering TS with master...
I20260811 19:14:49.917366 25796 heartbeater.cc:511] Master 127.3.61.190:42673 requested a full tablet report, sending...
I20260811 19:14:49.918941 25485 ts_manager.cc:194] Registered new tserver with Master: e5ebae3dbafb413c82ec94f8cf6bd1fb (127.3.61.129:38499)
I20260811 19:14:49.920261 25485 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.129@KRBTEST.COM'} at 127.3.61.129:53159
Aug 11 19:14:49 dist-test-slave-t7g5 krb5kdc[23924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475689, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.190@KRBTEST.COM for HTTP/127.3.61.148@KRBTEST.COM
W20260811 19:14:50.088433 26006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:14:50.088963 26006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:14:50.089177 26006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:14:50.101172 26006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:14:50.101385 26006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:14:50.101585 26006 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.131
I20260811 19:14:50.118359 26006 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:43131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.3.61.148:34415/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab
--principal=kudu/127.3.61.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.131:40539
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.61.131
--webserver_port=45315
--tserver_master_addrs=127.3.61.190:42673
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.131
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:14:50.120370 26006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:14:50.122577 26006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:14:50.146311 26013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:14:50.146641 26014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:14:50.149178 26017 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:14:50.150136 26006 server_base.cc:1034] running on GCE node
I20260811 19:14:50.150903 26006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:14:50.152050 26006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:14:50.153337 26006 hybrid_clock.cc:648] HybridClock initialized: now 1786475690153271 us; error 44 us; skew 500 ppm
I20260811 19:14:50.153764 26006 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:14:50 dist-test-slave-t7g5 krb5kdc[23924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475690, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:14:50.157192 26006 init.cc:377] Logged in from keytab as kudu/127.3.61.131@KRBTEST.COM (short username kudu)
I20260811 19:14:50.162361 26006 webserver.cc:460] Webserver started at http://127.3.61.131:45315/ using document root <none> and password file <none>
I20260811 19:14:50.163206 26006 fs_manager.cc:359] Metadata directory not provided
I20260811 19:14:50.163409 26006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 19:14:50 dist-test-slave-t7g5 krb5kdc[23924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475690, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.131@KRBTEST.COM for HTTP/127.3.61.148@KRBTEST.COM
I20260811 19:14:50.203969 26006 fs_manager.cc:709] Time spent opening directory manager: real 0.017s user 0.005s sys 0.001s
I20260811 19:14:50.208285 26028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:14:50.210441 26006 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 19:14:50.210665 26006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-2/wal
uuid: "c136675655204b019dd2894c3b373522"
format_stamp: "Formatted at 2026-08-11 19:14:46 on dist-test-slave-t7g5"
server_key: "804a6705690df33b5f0dcb24b74789b8"
server_key_iv: "7b0783c792295d850dd44c1152b4a2f3"
server_key_version: "kuduclusterkey@0"
I20260811 19:14:50.212252 26006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:14:50.240839 26006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:14:50.246933 26006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:14:50.247319 26006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:14:50.248708 26006 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:14:50.254611 26006 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:14:50.254742 26006 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:14:50.254875 26006 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:14:50.254956 26006 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 19:14:50 dist-test-slave-t7g5 krb5kdc[23924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475689, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.190@KRBTEST.COM for HTTP/127.3.61.148@KRBTEST.COM
I20260811 19:14:50.422693 26006 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.131:40539
I20260811 19:14:50.422729 26194 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.131:40539 every 8 connection(s)
I20260811 19:14:50.424738 26006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
I20260811 19:14:50.433882 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 26006
Aug 11 19:14:50 dist-test-slave-t7g5 krb5kdc[23924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475690, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.131@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:14:50.473322 26201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:50.446737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51892 (local address 127.3.61.190:42673)
0811 19:14:50.451897 (+ 5160us) server_negotiation.cc:206] Beginning negotiation
0811 19:14:50.451904 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:14:50.451932 (+ 28us) server_negotiation.cc:407] Connection header received
0811 19:14:50.451986 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:14:50.451991 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:14:50.452082 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:14:50.452253 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:14:50.454880 (+ 2627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:50.456149 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:50.458385 (+ 2236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:50.459018 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:50.462329 (+ 3311us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:14:50.463229 (+ 900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:14:50.463237 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:14:50.463242 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:14:50.463283 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:14:50.467601 (+ 4318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:14:50.468708 (+ 1107us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:14:50.468713 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:14:50.468716 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:14:50.468903 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:14:50.469428 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:14:50.469432 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:14:50.469436 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:14:50.470179 (+ 743us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:14:50.472439 (+ 2260us) server_negotiation.cc:1035] Waiting for connection context
0811 19:14:50.472608 (+ 169us) server_negotiation.cc:299] Negotiation successful
0811 19:14:50.472885 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4979,"thread_start_us":5677,"threads_started":1}
I20260811 19:14:50.473788 26199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:50.429060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:45249 (local address 127.3.61.190:42673)
0811 19:14:50.446817 (+ 17757us) server_negotiation.cc:206] Beginning negotiation
0811 19:14:50.446825 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:14:50.451365 (+ 4540us) server_negotiation.cc:407] Connection header received
0811 19:14:50.451478 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:14:50.451484 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:14:50.451663 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:14:50.451804 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:14:50.456142 (+ 4338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:50.457257 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:50.459769 (+ 2512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:50.460786 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:50.460922 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:14:50.466158 (+ 5236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:14:50.466166 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:14:50.466170 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:14:50.466228 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:14:50.469767 (+ 3539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:14:50.470863 (+ 1096us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:14:50.470869 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:14:50.470872 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:14:50.471009 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:14:50.471561 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:14:50.471566 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:14:50.471569 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:14:50.471932 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:14:50.472059 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 19:14:50.473144 (+ 1085us) server_negotiation.cc:299] Negotiation successful
0811 19:14:50.473360 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17597,"thread_start_us":17406,"threads_started":1}
I20260811 19:14:50.476202 26198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:50.428341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:42673 (local address 127.3.61.131:45249)
0811 19:14:50.447681 (+ 19340us) negotiation.cc:107] Waiting for socket to connect
0811 19:14:50.447780 (+ 99us) client_negotiation.cc:174] Beginning negotiation
0811 19:14:50.450781 (+ 3001us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:14:50.454188 (+ 3407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:14:50.454205 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:14:50.454907 (+ 702us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:14:50.455827 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:14:50.455863 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:14:50.457481 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:14:50.457486 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:14:50.459601 (+ 2115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:14:50.459618 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:14:50.461429 (+ 1811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:14:50.461438 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:14:50.461846 (+ 408us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:14:50.462694 (+ 848us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:14:50.462744 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:14:50.465814 (+ 3070us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:14:50.470033 (+ 4219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:14:50.470044 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:14:50.470066 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:14:50.470706 (+ 640us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:14:50.471150 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:14:50.471155 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:14:50.471158 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:14:50.471433 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:14:50.472159 (+ 726us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:14:50.472168 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:14:50.472661 (+ 493us) client_negotiation.cc:769] Sending connection context
0811 19:14:50.473990 (+ 1329us) client_negotiation.cc:240] Negotiation successful
0811 19:14:50.474623 (+ 633us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18864,"thread_start_us":18577,"threads_started":1}
I20260811 19:14:50.477885 26196 heartbeater.cc:347] Connected to a master server at 127.3.61.190:42673
I20260811 19:14:50.478236 26196 heartbeater.cc:464] Registering TS with master...
I20260811 19:14:50.479187 26196 heartbeater.cc:511] Master 127.3.61.190:42673 requested a full tablet report, sending...
I20260811 19:14:50.481406 25485 ts_manager.cc:194] Registered new tserver with Master: c136675655204b019dd2894c3b373522 (127.3.61.131:40539)
I20260811 19:14:50.482625 25485 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.131@KRBTEST.COM'} at 127.3.61.131:45249
I20260811 19:14:50.489415 3318 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 19:14:50.518144 26201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:50.492917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51894 (local address 127.3.61.190:42673)
0811 19:14:50.493113 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 19:14:50.493119 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:14:50.500220 (+ 7101us) server_negotiation.cc:407] Connection header received
0811 19:14:50.500842 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:14:50.500848 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:14:50.500953 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:14:50.501039 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:14:50.503459 (+ 2420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:50.504576 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:50.506856 (+ 2280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:50.507536 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:50.507656 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:14:50.511110 (+ 3454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:14:50.511118 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:14:50.511123 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:14:50.511174 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:14:50.514827 (+ 3653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:14:50.515732 (+ 905us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:14:50.515738 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:14:50.515743 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:14:50.515884 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:14:50.516286 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:14:50.516292 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:14:50.516296 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:14:50.516602 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:14:50.516846 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0811 19:14:50.517408 (+ 562us) server_negotiation.cc:299] Negotiation successful
0811 19:14:50.517636 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 19:14:50.525063 25485 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51894:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 19:14:50.536303 25485 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:14:50.584187 26226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:50.562845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:38499 (local address 127.0.0.1:59432)
0811 19:14:50.572474 (+ 9629us) negotiation.cc:107] Waiting for socket to connect
0811 19:14:50.572530 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 19:14:50.572774 (+ 244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:14:50.573986 (+ 1212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:14:50.573994 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:14:50.574032 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:14:50.574567 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:14:50.574581 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:14:50.577016 (+ 2435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:14:50.577021 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:14:50.580328 (+ 3307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:14:50.580356 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:14:50.583012 (+ 2656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:14:50.583020 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:14:50.583278 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:14:50.583343 (+ 65us) client_negotiation.cc:769] Sending connection context
0811 19:14:50.583495 (+ 152us) client_negotiation.cc:240] Negotiation successful
0811 19:14:50.583745 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9341,"thread_start_us":9171,"threads_started":1}
I20260811 19:14:50.585479 26229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:50.563481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59432 (local address 127.3.61.129:38499)
0811 19:14:50.571777 (+ 8296us) server_negotiation.cc:206] Beginning negotiation
0811 19:14:50.571789 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 19:14:50.572995 (+ 1206us) server_negotiation.cc:407] Connection header received
0811 19:14:50.573122 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:14:50.573130 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:14:50.573395 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:14:50.573595 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:14:50.574863 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:50.576845 (+ 1982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:50.580617 (+ 3772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:50.582328 (+ 1711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:50.582501 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:14:50.582789 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0811 19:14:50.584904 (+ 2115us) server_negotiation.cc:299] Negotiation successful
0811 19:14:50.585122 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8076,"thread_start_us":7903,"threads_started":1}
I20260811 19:14:50.599879 25686 tablet_service.cc:1467] Processing CreateTablet for tablet e3f57220bd2c40e19889b0063b6a18d9 (DEFAULT_TABLE table=test-table [id=7654267334c847e6896044c5b04b3a7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:14:50.601679 25686 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3f57220bd2c40e19889b0063b6a18d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:14:50.604584 26001 heartbeater.cc:503] Master 127.3.61.190:42673 was elected leader, sending a full tablet report...
I20260811 19:14:50.613255 26230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:50.576946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.131:40539 (local address 127.0.0.1:38724)
0811 19:14:50.593121 (+ 16175us) negotiation.cc:107] Waiting for socket to connect
0811 19:14:50.593145 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 19:14:50.593862 (+ 717us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:14:50.595914 (+ 2052us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:14:50.595919 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:14:50.595949 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:14:50.596325 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:14:50.596339 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:14:50.608223 (+ 11884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:14:50.608228 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:14:50.609948 (+ 1720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:14:50.609969 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:14:50.612380 (+ 2411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:14:50.612387 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:14:50.612578 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:14:50.612617 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 19:14:50.612716 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 19:14:50.612875 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15986,"thread_start_us":16803,"threads_started":1}
I20260811 19:14:50.617337 26235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:50.573458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38724 (local address 127.3.61.131:40539)
0811 19:14:50.594444 (+ 20986us) server_negotiation.cc:206] Beginning negotiation
0811 19:14:50.594452 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:14:50.594492 (+ 40us) server_negotiation.cc:407] Connection header received
0811 19:14:50.594603 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:14:50.594611 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:14:50.594932 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:14:50.595144 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:14:50.605816 (+ 10672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:50.607993 (+ 2177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:50.610173 (+ 2180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:50.611761 (+ 1588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:50.611923 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:14:50.612183 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0811 19:14:50.613572 (+ 1389us) server_negotiation.cc:299] Negotiation successful
0811 19:14:50.613789 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20784,"thread_start_us":20642,"threads_started":1}
I20260811 19:14:50.619656 26226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:50.571448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.130:36265 (local address 127.0.0.1:43040)
0811 19:14:50.584504 (+ 13056us) negotiation.cc:107] Waiting for socket to connect
0811 19:14:50.584528 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 19:14:50.584666 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:14:50.593288 (+ 8622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:14:50.593294 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:14:50.593340 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:14:50.593594 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:14:50.593613 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:14:50.614519 (+ 20906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:14:50.614525 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:14:50.616374 (+ 1849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:14:50.616394 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:14:50.618838 (+ 2444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:14:50.618844 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:14:50.619039 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:14:50.619084 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 19:14:50.619196 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 19:14:50.619366 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12930,"thread_start_us":21278,"threads_started":1}
I20260811 19:14:50.619885 26232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:50.571861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43040 (local address 127.3.61.130:36265)
0811 19:14:50.591082 (+ 19221us) server_negotiation.cc:206] Beginning negotiation
0811 19:14:50.591096 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 19:14:50.591183 (+ 87us) server_negotiation.cc:407] Connection header received
0811 19:14:50.591365 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:14:50.591376 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:14:50.591824 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:14:50.593320 (+ 1496us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:14:50.594057 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:50.595749 (+ 1692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:50.616626 (+ 20877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:50.618364 (+ 1738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:50.618530 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:14:50.618799 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0811 19:14:50.619286 (+ 487us) server_negotiation.cc:299] Negotiation successful
0811 19:14:50.619474 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18903,"thread_start_us":20368,"threads_started":1}
I20260811 19:14:50.622249 26104 tablet_service.cc:1467] Processing CreateTablet for tablet e3f57220bd2c40e19889b0063b6a18d9 (DEFAULT_TABLE table=test-table [id=7654267334c847e6896044c5b04b3a7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:14:50.623561 26104 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3f57220bd2c40e19889b0063b6a18d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:14:50.623972 25910 tablet_service.cc:1467] Processing CreateTablet for tablet e3f57220bd2c40e19889b0063b6a18d9 (DEFAULT_TABLE table=test-table [id=7654267334c847e6896044c5b04b3a7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:14:50.625440 25910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3f57220bd2c40e19889b0063b6a18d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:14:50.643039 26244 tablet_bootstrap.cc:492] T e3f57220bd2c40e19889b0063b6a18d9 P e5ebae3dbafb413c82ec94f8cf6bd1fb: Bootstrap starting.
I20260811 19:14:50.645180 26246 tablet_bootstrap.cc:492] T e3f57220bd2c40e19889b0063b6a18d9 P ea56db5fc5d343fbaedbeae56b958918: Bootstrap starting.
I20260811 19:14:50.652856 26244 tablet_bootstrap.cc:654] T e3f57220bd2c40e19889b0063b6a18d9 P e5ebae3dbafb413c82ec94f8cf6bd1fb: Neither blocks nor log segments found. Creating new log.
I20260811 19:14:50.654109 26240 tablet_bootstrap.cc:492] T e3f57220bd2c40e19889b0063b6a18d9 P c136675655204b019dd2894c3b373522: Bootstrap starting.
I20260811 19:14:50.654489 26244 log.cc:824] T e3f57220bd2c40e19889b0063b6a18d9 P e5ebae3dbafb413c82ec94f8cf6bd1fb: Log is configured to *not* fsync() on all Append() calls
I20260811 19:14:50.659020 26240 tablet_bootstrap.cc:654] T e3f57220bd2c40e19889b0063b6a18d9 P c136675655204b019dd2894c3b373522: Neither blocks nor log segments found. Creating new log.
I20260811 19:14:50.660573 26240 log.cc:824] T e3f57220bd2c40e19889b0063b6a18d9 P c136675655204b019dd2894c3b373522: Log is configured to *not* fsync() on all Append() calls
I20260811 19:14:50.665397 26246 tablet_bootstrap.cc:654] T e3f57220bd2c40e19889b0063b6a18d9 P ea56db5fc5d343fbaedbeae56b958918: Neither blocks nor log segments found. Creating new log.
I20260811 19:14:50.666934 26246 log.cc:824] T e3f57220bd2c40e19889b0063b6a18d9 P ea56db5fc5d343fbaedbeae56b958918: Log is configured to *not* fsync() on all Append() calls
I20260811 19:14:50.667752 26240 tablet_bootstrap.cc:492] T e3f57220bd2c40e19889b0063b6a18d9 P c136675655204b019dd2894c3b373522: No bootstrap required, opened a new log
I20260811 19:14:50.668130 26240 ts_tablet_manager.cc:1397] T e3f57220bd2c40e19889b0063b6a18d9 P c136675655204b019dd2894c3b373522: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.000s
I20260811 19:14:50.672664 26244 tablet_bootstrap.cc:492] T e3f57220bd2c40e19889b0063b6a18d9 P e5ebae3dbafb413c82ec94f8cf6bd1fb: No bootstrap required, opened a new log
I20260811 19:14:50.673035 26244 ts_tablet_manager.cc:1397] T e3f57220bd2c40e19889b0063b6a18d9 P e5ebae3dbafb413c82ec94f8cf6bd1fb: Time spent bootstrapping tablet: real 0.030s user 0.014s sys 0.010s
I20260811 19:14:50.676800 26246 tablet_bootstrap.cc:492] T e3f57220bd2c40e19889b0063b6a18d9 P ea56db5fc5d343fbaedbeae56b958918: No bootstrap required, opened a new log
I20260811 19:14:50.677239 26246 ts_tablet_manager.cc:1397] T e3f57220bd2c40e19889b0063b6a18d9 P ea56db5fc5d343fbaedbeae56b958918: Time spent bootstrapping tablet: real 0.033s user 0.010s sys 0.000s
I20260811 19:14:50.679001 26240 raft_consensus.cc:348] T e3f57220bd2c40e19889b0063b6a18d9 P c136675655204b019dd2894c3b373522 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5ebae3dbafb413c82ec94f8cf6bd1fb" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 38499 } } peers { permanent_uuid: "c136675655204b019dd2894c3b373522" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 40539 } } peers { permanent_uuid: "ea56db5fc5d343fbaedbeae56b958918" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 36265 } }
I20260811 19:14:50.679487 26240 raft_consensus.cc:374] T e3f57220bd2c40e19889b0063b6a18d9 P c136675655204b019dd2894c3b373522 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:14:50.679586 26240 raft_consensus.cc:729] T e3f57220bd2c40e19889b0063b6a18d9 P c136675655204b019dd2894c3b373522 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c136675655204b019dd2894c3b373522, State: Initialized, Role: FOLLOWER
I20260811 19:14:50.680353 26240 consensus_queue.cc:260] T e3f57220bd2c40e19889b0063b6a18d9 P c136675655204b019dd2894c3b373522 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5ebae3dbafb413c82ec94f8cf6bd1fb" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 38499 } } peers { permanent_uuid: "c136675655204b019dd2894c3b373522" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 40539 } } peers { permanent_uuid: "ea56db5fc5d343fbaedbeae56b958918" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 36265 } }
I20260811 19:14:50.684142 26244 raft_consensus.cc:348] T e3f57220bd2c40e19889b0063b6a18d9 P e5ebae3dbafb413c82ec94f8cf6bd1fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5ebae3dbafb413c82ec94f8cf6bd1fb" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 38499 } } peers { permanent_uuid: "c136675655204b019dd2894c3b373522" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 40539 } } peers { permanent_uuid: "ea56db5fc5d343fbaedbeae56b958918" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 36265 } }
I20260811 19:14:50.684700 26244 raft_consensus.cc:374] T e3f57220bd2c40e19889b0063b6a18d9 P e5ebae3dbafb413c82ec94f8cf6bd1fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:14:50.684823 26244 raft_consensus.cc:729] T e3f57220bd2c40e19889b0063b6a18d9 P e5ebae3dbafb413c82ec94f8cf6bd1fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5ebae3dbafb413c82ec94f8cf6bd1fb, State: Initialized, Role: FOLLOWER
I20260811 19:14:50.685694 26244 consensus_queue.cc:260] T e3f57220bd2c40e19889b0063b6a18d9 P e5ebae3dbafb413c82ec94f8cf6bd1fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5ebae3dbafb413c82ec94f8cf6bd1fb" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 38499 } } peers { permanent_uuid: "c136675655204b019dd2894c3b373522" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 40539 } } peers { permanent_uuid: "ea56db5fc5d343fbaedbeae56b958918" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 36265 } }
I20260811 19:14:50.689254 26196 heartbeater.cc:503] Master 127.3.61.190:42673 was elected leader, sending a full tablet report...
I20260811 19:14:50.690378 26240 ts_tablet_manager.cc:1428] T e3f57220bd2c40e19889b0063b6a18d9 P c136675655204b019dd2894c3b373522: Time spent starting tablet: real 0.022s user 0.012s sys 0.011s
I20260811 19:14:50.692289 25796 heartbeater.cc:503] Master 127.3.61.190:42673 was elected leader, sending a full tablet report...
I20260811 19:14:50.692617 26246 raft_consensus.cc:348] T e3f57220bd2c40e19889b0063b6a18d9 P ea56db5fc5d343fbaedbeae56b958918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5ebae3dbafb413c82ec94f8cf6bd1fb" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 38499 } } peers { permanent_uuid: "c136675655204b019dd2894c3b373522" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 40539 } } peers { permanent_uuid: "ea56db5fc5d343fbaedbeae56b958918" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 36265 } }
I20260811 19:14:50.693305 26244 ts_tablet_manager.cc:1428] T e3f57220bd2c40e19889b0063b6a18d9 P e5ebae3dbafb413c82ec94f8cf6bd1fb: Time spent starting tablet: real 0.020s user 0.011s sys 0.007s
I20260811 19:14:50.693302 26246 raft_consensus.cc:374] T e3f57220bd2c40e19889b0063b6a18d9 P ea56db5fc5d343fbaedbeae56b958918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:14:50.693565 26246 raft_consensus.cc:729] T e3f57220bd2c40e19889b0063b6a18d9 P ea56db5fc5d343fbaedbeae56b958918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea56db5fc5d343fbaedbeae56b958918, State: Initialized, Role: FOLLOWER
I20260811 19:14:50.694839 26246 consensus_queue.cc:260] T e3f57220bd2c40e19889b0063b6a18d9 P ea56db5fc5d343fbaedbeae56b958918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5ebae3dbafb413c82ec94f8cf6bd1fb" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 38499 } } peers { permanent_uuid: "c136675655204b019dd2894c3b373522" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 40539 } } peers { permanent_uuid: "ea56db5fc5d343fbaedbeae56b958918" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 36265 } }
I20260811 19:14:50.700034 26246 ts_tablet_manager.cc:1428] T e3f57220bd2c40e19889b0063b6a18d9 P ea56db5fc5d343fbaedbeae56b958918: Time spent starting tablet: real 0.023s user 0.006s sys 0.010s
W20260811 19:14:50.815524 26003 tablet.cc:2369] T e3f57220bd2c40e19889b0063b6a18d9 P ea56db5fc5d343fbaedbeae56b958918: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:14:50.831880 26262 raft_consensus.cc:482] T e3f57220bd2c40e19889b0063b6a18d9 P c136675655204b019dd2894c3b373522 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:14:50.832357 26262 raft_consensus.cc:504] T e3f57220bd2c40e19889b0063b6a18d9 P c136675655204b019dd2894c3b373522 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5ebae3dbafb413c82ec94f8cf6bd1fb" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 38499 } } peers { permanent_uuid: "c136675655204b019dd2894c3b373522" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 40539 } } peers { permanent_uuid: "ea56db5fc5d343fbaedbeae56b958918" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 36265 } }
I20260811 19:14:50.835148 26262 leader_election.cc:288] T e3f57220bd2c40e19889b0063b6a18d9 P c136675655204b019dd2894c3b373522 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5ebae3dbafb413c82ec94f8cf6bd1fb (127.3.61.129:38499), ea56db5fc5d343fbaedbeae56b958918 (127.3.61.130:36265)
I20260811 19:14:50.843894 26198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:50.835865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:38499 (local address 127.3.61.131:42269)
0811 19:14:50.836362 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0811 19:14:50.836385 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 19:14:50.836541 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:14:50.837541 (+ 1000us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:14:50.837546 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:14:50.837575 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:14:50.837818 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:14:50.837830 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:14:50.839424 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:14:50.839431 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:14:50.841486 (+ 2055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:14:50.841506 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:14:50.843057 (+ 1551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:14:50.843064 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:14:50.843287 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:14:50.843325 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 19:14:50.843431 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 19:14:50.843588 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350}
I20260811 19:14:50.843909 26229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:50.836833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:42269 (local address 127.3.61.129:38499)
0811 19:14:50.837052 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 19:14:50.837058 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:14:50.837080 (+ 22us) server_negotiation.cc:407] Connection header received
0811 19:14:50.837152 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:14:50.837157 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:14:50.837248 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:14:50.837359 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:14:50.837987 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:50.839189 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:50.841764 (+ 2575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:50.842863 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:50.843045 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:14:50.843125 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 19:14:50.843483 (+ 358us) server_negotiation.cc:299] Negotiation successful
0811 19:14:50.843629 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 19:14:50.845320 25713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3f57220bd2c40e19889b0063b6a18d9" candidate_uuid: "c136675655204b019dd2894c3b373522" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5ebae3dbafb413c82ec94f8cf6bd1fb" is_pre_election: true
I20260811 19:14:50.846050 25713 raft_consensus.cc:2454] T e3f57220bd2c40e19889b0063b6a18d9 P e5ebae3dbafb413c82ec94f8cf6bd1fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c136675655204b019dd2894c3b373522 in term 0.
I20260811 19:14:50.846912 26029 leader_election.cc:302] T e3f57220bd2c40e19889b0063b6a18d9 P c136675655204b019dd2894c3b373522 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c136675655204b019dd2894c3b373522, e5ebae3dbafb413c82ec94f8cf6bd1fb; no voters:
I20260811 19:14:50.847373 26262 raft_consensus.cc:2781] T e3f57220bd2c40e19889b0063b6a18d9 P c136675655204b019dd2894c3b373522 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:14:50.847512 26262 raft_consensus.cc:482] T e3f57220bd2c40e19889b0063b6a18d9 P c136675655204b019dd2894c3b373522 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:14:50.847625 26262 raft_consensus.cc:3037] T e3f57220bd2c40e19889b0063b6a18d9 P c136675655204b019dd2894c3b373522 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:14:50.848891 26232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:50.836706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:50383 (local address 127.3.61.130:36265)
0811 19:14:50.836924 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 19:14:50.836930 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:14:50.842189 (+ 5259us) server_negotiation.cc:407] Connection header received
0811 19:14:50.842342 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:14:50.842348 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:14:50.842437 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:14:50.842545 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:14:50.843224 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:50.844201 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:50.846471 (+ 2270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:50.847759 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:50.847959 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:14:50.848041 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 19:14:50.848461 (+ 420us) server_negotiation.cc:299] Negotiation successful
0811 19:14:50.848614 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 19:14:50.849134 26266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:50.836224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.130:36265 (local address 127.3.61.131:50383)
0811 19:14:50.842040 (+ 5816us) negotiation.cc:107] Waiting for socket to connect
0811 19:14:50.842061 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 19:14:50.842207 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:14:50.842579 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:14:50.842584 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:14:50.842613 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:14:50.843012 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:14:50.843028 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:14:50.844405 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:14:50.844411 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:14:50.846247 (+ 1836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:14:50.846267 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:14:50.848006 (+ 1739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:14:50.848012 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:14:50.848213 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:14:50.848254 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 19:14:50.848410 (+ 156us) client_negotiation.cc:240] Negotiation successful
0811 19:14:50.848570 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5673,"thread_start_us":5449,"threads_started":1}
I20260811 19:14:50.850531 25937 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3f57220bd2c40e19889b0063b6a18d9" candidate_uuid: "c136675655204b019dd2894c3b373522" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea56db5fc5d343fbaedbeae56b958918" is_pre_election: true
I20260811 19:14:50.850749 26262 raft_consensus.cc:504] T e3f57220bd2c40e19889b0063b6a18d9 P c136675655204b019dd2894c3b373522 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5ebae3dbafb413c82ec94f8cf6bd1fb" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 38499 } } peers { permanent_uuid: "c136675655204b019dd2894c3b373522" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 40539 } } peers { permanent_uuid: "ea56db5fc5d343fbaedbeae56b958918" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 36265 } }
I20260811 19:14:50.851135 25937 raft_consensus.cc:2454] T e3f57220bd2c40e19889b0063b6a18d9 P ea56db5fc5d343fbaedbeae56b958918 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c136675655204b019dd2894c3b373522 in term 0.
I20260811 19:14:50.851533 26262 leader_election.cc:288] T e3f57220bd2c40e19889b0063b6a18d9 P c136675655204b019dd2894c3b373522 [CANDIDATE]: Term 1 election: Requested vote from peers e5ebae3dbafb413c82ec94f8cf6bd1fb (127.3.61.129:38499), ea56db5fc5d343fbaedbeae56b958918 (127.3.61.130:36265)
I20260811 19:14:50.852172 25937 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3f57220bd2c40e19889b0063b6a18d9" candidate_uuid: "c136675655204b019dd2894c3b373522" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea56db5fc5d343fbaedbeae56b958918"
I20260811 19:14:50.852159 25713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3f57220bd2c40e19889b0063b6a18d9" candidate_uuid: "c136675655204b019dd2894c3b373522" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5ebae3dbafb413c82ec94f8cf6bd1fb"
I20260811 19:14:50.852403 25937 raft_consensus.cc:3037] T e3f57220bd2c40e19889b0063b6a18d9 P ea56db5fc5d343fbaedbeae56b958918 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:14:50.852403 25713 raft_consensus.cc:3037] T e3f57220bd2c40e19889b0063b6a18d9 P e5ebae3dbafb413c82ec94f8cf6bd1fb [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:14:50.855157 25713 raft_consensus.cc:2454] T e3f57220bd2c40e19889b0063b6a18d9 P e5ebae3dbafb413c82ec94f8cf6bd1fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c136675655204b019dd2894c3b373522 in term 1.
I20260811 19:14:50.855649 26029 leader_election.cc:302] T e3f57220bd2c40e19889b0063b6a18d9 P c136675655204b019dd2894c3b373522 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c136675655204b019dd2894c3b373522, e5ebae3dbafb413c82ec94f8cf6bd1fb; no voters:
I20260811 19:14:50.855835 25937 raft_consensus.cc:2454] T e3f57220bd2c40e19889b0063b6a18d9 P ea56db5fc5d343fbaedbeae56b958918 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c136675655204b019dd2894c3b373522 in term 1.
I20260811 19:14:50.855952 26262 raft_consensus.cc:2781] T e3f57220bd2c40e19889b0063b6a18d9 P c136675655204b019dd2894c3b373522 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:14:50.856637 26262 raft_consensus.cc:686] T e3f57220bd2c40e19889b0063b6a18d9 P c136675655204b019dd2894c3b373522 [term 1 LEADER]: Becoming Leader. State: Replica: c136675655204b019dd2894c3b373522, State: Running, Role: LEADER
I20260811 19:14:50.857259 26262 consensus_queue.cc:237] T e3f57220bd2c40e19889b0063b6a18d9 P c136675655204b019dd2894c3b373522 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5ebae3dbafb413c82ec94f8cf6bd1fb" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 38499 } } peers { permanent_uuid: "c136675655204b019dd2894c3b373522" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 40539 } } peers { permanent_uuid: "ea56db5fc5d343fbaedbeae56b958918" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 36265 } }
I20260811 19:14:50.863587 25485 catalog_manager.cc:5495] T e3f57220bd2c40e19889b0063b6a18d9 P c136675655204b019dd2894c3b373522 reported cstate change: term changed from 0 to 1, leader changed from <none> to c136675655204b019dd2894c3b373522 (127.3.61.131). New cstate: current_term: 1 leader_uuid: "c136675655204b019dd2894c3b373522" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5ebae3dbafb413c82ec94f8cf6bd1fb" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 38499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c136675655204b019dd2894c3b373522" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 40539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ea56db5fc5d343fbaedbeae56b958918" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 36265 } health_report { overall_health: UNKNOWN } } }
W20260811 19:14:50.883158 25797 tablet.cc:2369] T e3f57220bd2c40e19889b0063b6a18d9 P e5ebae3dbafb413c82ec94f8cf6bd1fb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:14:50.926486 26235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:50.919637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38736 (local address 127.3.61.131:40539)
0811 19:14:50.919839 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 19:14:50.919845 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:14:50.919866 (+ 21us) server_negotiation.cc:407] Connection header received
0811 19:14:50.920095 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:14:50.920100 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:14:50.920264 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:14:50.920459 (+ 195us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:14:50.921167 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:50.922292 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:50.923831 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:50.924400 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:50.924522 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:14:50.925109 (+ 587us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:14:50.925871 (+ 762us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:14:50.926011 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 19:14:50.926133 (+ 122us) server_negotiation.cc:299] Negotiation successful
0811 19:14:50.926220 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 19:14:50.936702 25937 raft_consensus.cc:1261] T e3f57220bd2c40e19889b0063b6a18d9 P ea56db5fc5d343fbaedbeae56b958918 [term 1 FOLLOWER]: Refusing update from remote peer c136675655204b019dd2894c3b373522: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:14:50.937049 25713 raft_consensus.cc:1261] T e3f57220bd2c40e19889b0063b6a18d9 P e5ebae3dbafb413c82ec94f8cf6bd1fb [term 1 FOLLOWER]: Refusing update from remote peer c136675655204b019dd2894c3b373522: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:14:50.938026 26262 consensus_queue.cc:1035] T e3f57220bd2c40e19889b0063b6a18d9 P c136675655204b019dd2894c3b373522 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea56db5fc5d343fbaedbeae56b958918" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 36265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:14:50.938367 26267 consensus_queue.cc:1035] T e3f57220bd2c40e19889b0063b6a18d9 P c136675655204b019dd2894c3b373522 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5ebae3dbafb413c82ec94f8cf6bd1fb" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 38499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 19:14:50.938961 26197 tablet.cc:2369] T e3f57220bd2c40e19889b0063b6a18d9 P c136675655204b019dd2894c3b373522: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:14:50.955016 26270 mvcc.cc:205] Tried to move back new op lower bound from 7317404430064758784 to 7317404429756030976. Current Snapshot: MvccSnapshot[applied={T|T < 7317404430064758784}]
I20260811 19:14:50.957204 26271 mvcc.cc:205] Tried to move back new op lower bound from 7317404430064758784 to 7317404429756030976. Current Snapshot: MvccSnapshot[applied={T|T < 7317404430064758784}]
I20260811 19:14:50.962430 26272 mvcc.cc:205] Tried to move back new op lower bound from 7317404430064758784 to 7317404429756030976. Current Snapshot: MvccSnapshot[applied={T|T < 7317404430064758784 or (T in {7317404430064758784})}]
I20260811 19:14:50.980055 25485 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51894:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:14:50.980356 25485 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51894:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:14:50.984680 25485 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 062240d5b8e64e53818993a208e7cd8a: Sending DeleteTablet for 3 replicas of tablet e3f57220bd2c40e19889b0063b6a18d9
I20260811 19:14:50.986065 25910 tablet_service.cc:1514] Processing DeleteTablet for tablet e3f57220bd2c40e19889b0063b6a18d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:14:50 UTC) from {username='kudu', principal='kudu/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:43040
I20260811 19:14:50.986167 26104 tablet_service.cc:1514] Processing DeleteTablet for tablet e3f57220bd2c40e19889b0063b6a18d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:14:50 UTC) from {username='kudu', principal='kudu/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:38724
I20260811 19:14:50.986438 25686 tablet_service.cc:1514] Processing DeleteTablet for tablet e3f57220bd2c40e19889b0063b6a18d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:14:50 UTC) from {username='kudu', principal='kudu/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:59432
I20260811 19:14:50.987462 26291 tablet_replica.cc:330] T e3f57220bd2c40e19889b0063b6a18d9 P e5ebae3dbafb413c82ec94f8cf6bd1fb: stopping tablet replica
I20260811 19:14:50.988070 26291 raft_consensus.cc:2229] T e3f57220bd2c40e19889b0063b6a18d9 P e5ebae3dbafb413c82ec94f8cf6bd1fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:14:50.988808 26291 raft_consensus.cc:2258] T e3f57220bd2c40e19889b0063b6a18d9 P e5ebae3dbafb413c82ec94f8cf6bd1fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:14:50.992265 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 25538
I20260811 19:14:50.992609 26291 ts_tablet_manager.cc:1905] T e3f57220bd2c40e19889b0063b6a18d9 P e5ebae3dbafb413c82ec94f8cf6bd1fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:14:51.001246 26291 ts_tablet_manager.cc:1918] T e3f57220bd2c40e19889b0063b6a18d9 P e5ebae3dbafb413c82ec94f8cf6bd1fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:14:51.001782 26291 log.cc:1196] T e3f57220bd2c40e19889b0063b6a18d9 P e5ebae3dbafb413c82ec94f8cf6bd1fb: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-0/wal/wals/e3f57220bd2c40e19889b0063b6a18d9
I20260811 19:14:51.002545 26291 ts_tablet_manager.cc:1939] T e3f57220bd2c40e19889b0063b6a18d9 P e5ebae3dbafb413c82ec94f8cf6bd1fb: Deleting consensus metadata
I20260811 19:14:51.005179 25456 catalog_manager.cc:4841] TS e5ebae3dbafb413c82ec94f8cf6bd1fb (127.3.61.129:38499): tablet e3f57220bd2c40e19889b0063b6a18d9 (table test-table [id=7654267334c847e6896044c5b04b3a7f]) successfully deleted
I20260811 19:14:51.010499 26289 tablet_replica.cc:330] T e3f57220bd2c40e19889b0063b6a18d9 P c136675655204b019dd2894c3b373522: stopping tablet replica
I20260811 19:14:51.011030 26289 raft_consensus.cc:2229] T e3f57220bd2c40e19889b0063b6a18d9 P c136675655204b019dd2894c3b373522 [term 1 LEADER]: Raft consensus shutting down.
I20260811 19:14:51.011852 26289 raft_consensus.cc:2258] T e3f57220bd2c40e19889b0063b6a18d9 P c136675655204b019dd2894c3b373522 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:14:51.012573 26288 tablet_replica.cc:330] T e3f57220bd2c40e19889b0063b6a18d9 P ea56db5fc5d343fbaedbeae56b958918: stopping tablet replica
I20260811 19:14:51.014005 26288 raft_consensus.cc:2229] T e3f57220bd2c40e19889b0063b6a18d9 P ea56db5fc5d343fbaedbeae56b958918 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:14:51.014297 26289 ts_tablet_manager.cc:1905] T e3f57220bd2c40e19889b0063b6a18d9 P c136675655204b019dd2894c3b373522: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:14:51.014858 26288 raft_consensus.cc:2258] T e3f57220bd2c40e19889b0063b6a18d9 P ea56db5fc5d343fbaedbeae56b958918 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:14:51.017554 26288 ts_tablet_manager.cc:1905] T e3f57220bd2c40e19889b0063b6a18d9 P ea56db5fc5d343fbaedbeae56b958918: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:14:51.025966 26288 ts_tablet_manager.cc:1918] T e3f57220bd2c40e19889b0063b6a18d9 P ea56db5fc5d343fbaedbeae56b958918: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:14:51.026291 26288 log.cc:1196] T e3f57220bd2c40e19889b0063b6a18d9 P ea56db5fc5d343fbaedbeae56b958918: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-1/wal/wals/e3f57220bd2c40e19889b0063b6a18d9
I20260811 19:14:51.026980 26288 ts_tablet_manager.cc:1939] T e3f57220bd2c40e19889b0063b6a18d9 P ea56db5fc5d343fbaedbeae56b958918: Deleting consensus metadata
I20260811 19:14:51.028199 26229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:50.993768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59444 (local address 127.3.61.129:38499)
0811 19:14:50.993930 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 19:14:50.993937 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:14:51.006288 (+ 12351us) server_negotiation.cc:407] Connection header received
0811 19:14:51.006597 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:14:51.006604 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:14:51.006701 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:14:51.006779 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:14:51.009111 (+ 2332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:51.010464 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:51.013558 (+ 3094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:51.014317 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:51.014604 (+ 287us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:14:51.019233 (+ 4629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:14:51.019258 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:14:51.019292 (+ 34us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:14:51.019354 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:14:51.023863 (+ 4509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:14:51.025120 (+ 1257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:14:51.025133 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:14:51.025137 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:14:51.025333 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:14:51.025926 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:14:51.025931 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:14:51.025935 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:14:51.026496 (+ 561us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:14:51.026798 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0811 19:14:51.027503 (+ 705us) server_negotiation.cc:299] Negotiation successful
0811 19:14:51.027788 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 19:14:51.029567 26289 ts_tablet_manager.cc:1918] T e3f57220bd2c40e19889b0063b6a18d9 P c136675655204b019dd2894c3b373522: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:14:51.029564 25459 catalog_manager.cc:4841] TS ea56db5fc5d343fbaedbeae56b958918 (127.3.61.130:36265): tablet e3f57220bd2c40e19889b0063b6a18d9 (table test-table [id=7654267334c847e6896044c5b04b3a7f]) successfully deleted
I20260811 19:14:51.029862 25785 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:14:51.029884 26289 log.cc:1196] T e3f57220bd2c40e19889b0063b6a18d9 P c136675655204b019dd2894c3b373522: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786475454804547-3318-0/minicluster-data/ts-2/wal/wals/e3f57220bd2c40e19889b0063b6a18d9
I20260811 19:14:51.030462 26289 ts_tablet_manager.cc:1939] T e3f57220bd2c40e19889b0063b6a18d9 P c136675655204b019dd2894c3b373522: Deleting consensus metadata
I20260811 19:14:51.032415 25456 catalog_manager.cc:4841] TS c136675655204b019dd2894c3b373522 (127.3.61.131:40539): tablet e3f57220bd2c40e19889b0063b6a18d9 (table test-table [id=7654267334c847e6896044c5b04b3a7f]) successfully deleted
I20260811 19:14:51.162142 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 25538
I20260811 19:14:51.198536 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 25800
I20260811 19:14:51.217630 26232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:51.199481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43050 (local address 127.3.61.130:36265)
0811 19:14:51.199675 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 19:14:51.199681 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:14:51.199813 (+ 132us) server_negotiation.cc:407] Connection header received
0811 19:14:51.200000 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:14:51.200007 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:14:51.200094 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:14:51.200205 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:14:51.202127 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:51.203201 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:51.205175 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:51.205882 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:51.206019 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:14:51.209719 (+ 3700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:14:51.209745 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:14:51.209771 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:14:51.209838 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:14:51.213832 (+ 3994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:14:51.215006 (+ 1174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:14:51.215017 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:14:51.215020 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:14:51.215277 (+ 257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:14:51.215752 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:14:51.215758 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:14:51.215761 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:14:51.216321 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:14:51.216634 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0811 19:14:51.216963 (+ 329us) server_negotiation.cc:299] Negotiation successful
0811 19:14:51.217215 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 19:14:51.218408 25994 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:14:51.320308 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 25800
I20260811 19:14:51.348505 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 26006
I20260811 19:14:51.366961 26235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:51.349347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38740 (local address 127.3.61.131:40539)
0811 19:14:51.349564 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 19:14:51.349570 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:14:51.349906 (+ 336us) server_negotiation.cc:407] Connection header received
0811 19:14:51.350088 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:14:51.350092 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:14:51.350174 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:14:51.350260 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:14:51.351583 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:51.352351 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:51.353821 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:51.354263 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:51.354411 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:14:51.358096 (+ 3685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:14:51.358121 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:14:51.358146 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:14:51.358222 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:14:51.362549 (+ 4327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:14:51.364247 (+ 1698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:14:51.364258 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:14:51.364261 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:14:51.364520 (+ 259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:14:51.364995 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:14:51.364998 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:14:51.365001 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:14:51.365546 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:14:51.365982 (+ 436us) server_negotiation.cc:1035] Waiting for connection context
0811 19:14:51.366308 (+ 326us) server_negotiation.cc:299] Negotiation successful
0811 19:14:51.366586 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 19:14:51.367983 26188 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:14:51.496079 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 26006
I20260811 19:14:51.532033 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 25428
I20260811 19:14:51.533418 25525 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:14:51.647385 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 25428
I20260811 19:14:51.689407 3318 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 19:14:52.037348 3318 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 19:14:52.037577 3318 mini_ranger.cc:66] Stopping Ranger...
I20260811 19:14:52.396543 3318 mini_ranger.cc:68] Stopped Ranger
2026-08-11 19:14:52.397 UTC [23977] LOG: received smart shutdown request
2026-08-11 19:14:52.401 UTC [23977] LOG: background worker "logical replication launcher" (PID 23991) exited with exit code 1
2026-08-11 19:14:52.401 UTC [23980] LOG: shutting down
2026-08-11 19:14:52.599 UTC [23977] LOG: database system is shut down
2026-08-11T19:14:52Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (59823 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:14:52 dist-test-slave-t7g5 krb5kdc[26308](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:14:52 dist-test-slave-t7g5 krb5kdc[26308](info): set up 2 sockets
Aug 11 19:14:52 dist-test-slave-t7g5 krb5kdc[26308](info): commencing operation
krb5kdc: starting...
W20260811 19:14:54.071673 3318 mini_kdc.cc:121] Time spent starting KDC: real 1.299s user 0.000s sys 0.056s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Aug 11 19:14:54 dist-test-slave-t7g5 krb5kdc[26308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:14:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:14:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.61.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab.
Entry for principal kudu/127.3.61.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab.
I20260811 19:14:54.430713 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:41645
--webserver_interface=127.3.61.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:34969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.61.190:41645,127.3.61.189:43763,127.3.61.188:43463
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:14:54.827512 26324 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:14:54.827869 26324 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:14:54.827970 26324 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:14:54.835951 26324 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:14:54.836141 26324 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:14:54.836225 26324 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:14:54.836318 26324 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:14:54.836383 26324 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:14:54.847255 26324 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:34969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.61.190:41645,127.3.61.189:43763,127.3.61.188:43463
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:41645
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.61.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:14:54.848812 26324 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:14:54.850339 26324 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:14:54.863103 26336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:14:54.863389 26337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:14:54.863502 26340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:14:54.864591 26324 server_base.cc:1034] running on GCE node
I20260811 19:14:54.865548 26324 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:14:54.867465 26324 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:14:54.868963 26324 hybrid_clock.cc:648] HybridClock initialized: now 1786475694868881 us; error 77 us; skew 500 ppm
I20260811 19:14:54.869415 26324 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:14:54 dist-test-slave-t7g5 krb5kdc[26308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475694, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:14:54.873790 26324 init.cc:377] Logged in from keytab as kudu/127.3.61.190@KRBTEST.COM (short username kudu)
I20260811 19:14:54.876190 26324 webserver.cc:460] Webserver started at http://127.3.61.190:34171/ using document root <none> and password file <none>
I20260811 19:14:54.877312 26324 fs_manager.cc:359] Metadata directory not provided
I20260811 19:14:54.877454 26324 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:14:54.877879 26324 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:14:54.882341 26324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/minicluster-data/master-0/data/instance:
uuid: "7ad5c7bfe7a14c3fb4408d6ed8a11057"
format_stamp: "Formatted at 2026-08-11 19:14:54 on dist-test-slave-t7g5"
server_key: "49593ec510587cf97e86a2f937b92623"
server_key_iv: "6d511f3ffbc8b07d5d67c752a6d9e278"
server_key_version: "encryptionkey@0"
I20260811 19:14:54.883404 26324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/minicluster-data/master-0/wal/instance:
uuid: "7ad5c7bfe7a14c3fb4408d6ed8a11057"
format_stamp: "Formatted at 2026-08-11 19:14:54 on dist-test-slave-t7g5"
server_key: "49593ec510587cf97e86a2f937b92623"
server_key_iv: "6d511f3ffbc8b07d5d67c752a6d9e278"
server_key_version: "encryptionkey@0"
I20260811 19:14:54.890767 26324 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260811 19:14:54.896229 26349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:14:54.898212 26324 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 19:14:54.898428 26324 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/minicluster-data/master-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/minicluster-data/master-0/wal
uuid: "7ad5c7bfe7a14c3fb4408d6ed8a11057"
format_stamp: "Formatted at 2026-08-11 19:14:54 on dist-test-slave-t7g5"
server_key: "49593ec510587cf97e86a2f937b92623"
server_key_iv: "6d511f3ffbc8b07d5d67c752a6d9e278"
server_key_version: "encryptionkey@0"
I20260811 19:14:54.898684 26324 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:14:54.921079 26324 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:14:54.927893 26324 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:14:54.928680 26324 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:14:54.948573 26324 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.190:41645
I20260811 19:14:54.948637 26428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.190:41645 every 8 connection(s)
I20260811 19:14:54.950208 26324 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
I20260811 19:14:54.954500 26429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:14:54.959455 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 26324
I20260811 19:14:54.959715 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/minicluster-data/master-0/wal/instance
I20260811 19:14:54.960619 3318 external_mini_cluster.cc:1428] Setting key 637314ef3a7256d354ac88d31d930c09
I20260811 19:14:54.963163 26429 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.61.190" port: 41645 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 19:14:54 dist-test-slave-t7g5 krb5kdc[26308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475694, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.190@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:14:54.993897 26432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:54.967271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36764 (local address 127.3.61.190:41645)
0811 19:14:54.969378 (+ 2107us) server_negotiation.cc:206] Beginning negotiation
0811 19:14:54.969388 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 19:14:54.969449 (+ 61us) server_negotiation.cc:407] Connection header received
0811 19:14:54.971134 (+ 1685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:14:54.971141 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:14:54.971401 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:14:54.971598 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:14:54.973002 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:54.974394 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:54.976519 (+ 2125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:54.977427 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:54.977634 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:14:54.982752 (+ 5118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:14:54.982762 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:14:54.982779 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:14:54.982834 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:14:54.986949 (+ 4115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:14:54.988302 (+ 1353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:14:54.988311 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:14:54.988346 (+ 35us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:14:54.988683 (+ 337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:14:54.989519 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:14:54.989525 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:14:54.989528 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:14:54.990397 (+ 869us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:14:54.990628 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0811 19:14:54.991788 (+ 1160us) server_negotiation.cc:299] Negotiation successful
0811 19:14:54.992229 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":1768,"spinlock_wait_cycles":20864,"thread_start_us":1524,"threads_started":1}
I20260811 19:14:54.993989 26431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:54.967271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:41645 (local address 127.0.0.1:36764)
0811 19:14:54.968265 (+ 994us) negotiation.cc:107] Waiting for socket to connect
0811 19:14:54.968319 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 19:14:54.970670 (+ 2351us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:14:54.971633 (+ 963us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:14:54.971640 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:14:54.972432 (+ 792us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:14:54.972796 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:14:54.972816 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:14:54.974597 (+ 1781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:14:54.974602 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:14:54.976336 (+ 1734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:14:54.976353 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:14:54.977768 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:14:54.977776 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:14:54.977937 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:14:54.978843 (+ 906us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:14:54.978884 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:14:54.982419 (+ 3535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:14:54.987231 (+ 4812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:14:54.987252 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:14:54.987273 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:14:54.987997 (+ 724us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:14:54.988871 (+ 874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:14:54.988877 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:14:54.988881 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:14:54.989306 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:14:54.990664 (+ 1358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:14:54.990678 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:14:54.991136 (+ 458us) client_negotiation.cc:769] Sending connection context
0811 19:14:54.991589 (+ 453us) client_negotiation.cc:240] Negotiation successful
0811 19:14:54.992509 (+ 920us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":701,"mutex_wait_us":210,"thread_start_us":430,"threads_started":1}
WARNING: no policy specified for kudu/127.3.61.189@KRBTEST.COM; defaulting to no policy
I20260811 19:14:54.997718 26429 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43763 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.189@KRBTEST.COM" created.
I20260811 19:14:54.999330 26431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:54.998848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.189:43763 (local address 127.0.0.1:57218)
0811 19:14:54.999050 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 19:14:54.999183 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.189:43763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260811 19:14:54.999759 26352 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.61.189:43763: connect: Connection refused (error 111)
I20260811 19:14:55.002174 26431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:55.001495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.189:43763 (local address 127.0.0.1:57220)
0811 19:14:55.001834 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0811 19:14:55.001949 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.189:43763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
W20260811 19:14:55.002460 26429 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.61.189:43763: Network error: Client connection negotiation failed: client connection to 127.3.61.189:43763: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.189.keytab.
Entry for principal kudu/127.3.61.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.189.keytab.
I20260811 19:14:55.038914 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.189:43763
--webserver_interface=127.3.61.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.189.keytab
--principal=kudu/127.3.61.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:34969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.61.190:41645,127.3.61.189:43763,127.3.61.188:43463
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:14:55.052054 26429 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43763 } attempt: 1
I20260811 19:14:55.053732 26431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:55.053069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.189:43763 (local address 127.0.0.1:57228)
0811 19:14:55.053312 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0811 19:14:55.053457 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.189:43763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260811 19:14:55.055264 26431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:55.054783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.189:43763 (local address 127.0.0.1:57242)
0811 19:14:55.054960 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 19:14:55.055065 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.189:43763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260811 19:14:55.055583 26429 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.61.189:43763: Network error: Client connection negotiation failed: client connection to 127.3.61.189:43763: connect: Connection refused (error 111)
I20260811 19:14:55.123914 26429 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43763 } attempt: 2
I20260811 19:14:55.125569 26431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:55.124887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.189:43763 (local address 127.0.0.1:57252)
0811 19:14:55.125202 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0811 19:14:55.125332 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.189:43763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260811 19:14:55.127018 26431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:55.126522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.189:43763 (local address 127.0.0.1:57264)
0811 19:14:55.126721 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0811 19:14:55.126838 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.189:43763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260811 19:14:55.127288 26429 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.61.189:43763: Network error: Client connection negotiation failed: client connection to 127.3.61.189:43763: connect: Connection refused (error 111)
I20260811 19:14:55.218811 26429 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43763 } attempt: 3
I20260811 19:14:55.220564 26431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:55.219876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.189:43763 (local address 127.0.0.1:57274)
0811 19:14:55.220201 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0811 19:14:55.220370 (+ 169us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.189:43763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
I20260811 19:14:55.221917 26431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:55.221416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.189:43763 (local address 127.0.0.1:57278)
0811 19:14:55.221592 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 19:14:55.221723 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.189:43763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260811 19:14:55.222146 26429 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.61.189:43763: Network error: Client connection negotiation failed: client connection to 127.3.61.189:43763: connect: Connection refused (error 111)
I20260811 19:14:55.365468 26429 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43763 } attempt: 4
I20260811 19:14:55.367396 26431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:55.366664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.189:43763 (local address 127.0.0.1:57286)
0811 19:14:55.367052 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0811 19:14:55.367180 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.189:43763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207}
I20260811 19:14:55.368667 26431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:55.368154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.189:43763 (local address 127.0.0.1:57298)
0811 19:14:55.368339 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0811 19:14:55.368479 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.189:43763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260811 19:14:55.369138 26429 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.61.189:43763: Network error: Client connection negotiation failed: client connection to 127.3.61.189:43763: connect: Connection refused (error 111)
W20260811 19:14:55.436101 26439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:14:55.436481 26439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:14:55.436611 26439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:14:55.447687 26439 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:14:55.447849 26439 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:14:55.447907 26439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:14:55.447995 26439 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:14:55.448061 26439 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:14:55.459424 26439 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:34969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.3.61.190:41645,127.3.61.189:43763,127.3.61.188:43463
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.189.keytab
--principal=kudu/127.3.61.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.189:43763
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.3.61.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:14:55.460903 26439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:14:55.462337 26439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:14:55.478663 26448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:14:55.478772 26450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:14:55.478785 26447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:14:55.483559 26439 server_base.cc:1034] running on GCE node
I20260811 19:14:55.484858 26439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:14:55.486725 26439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:14:55.488111 26439 hybrid_clock.cc:648] HybridClock initialized: now 1786475695488053 us; error 95 us; skew 500 ppm
I20260811 19:14:55.488570 26439 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:14:55 dist-test-slave-t7g5 krb5kdc[26308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475695, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:14:55.492537 26439 init.cc:377] Logged in from keytab as kudu/127.3.61.189@KRBTEST.COM (short username kudu)
I20260811 19:14:55.494508 26439 webserver.cc:460] Webserver started at http://127.3.61.189:45889/ using document root <none> and password file <none>
I20260811 19:14:55.495343 26439 fs_manager.cc:359] Metadata directory not provided
I20260811 19:14:55.495494 26439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:14:55.495842 26439 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:14:55.499676 26439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/minicluster-data/master-1/data/instance:
uuid: "581dcb09f4374097938f97b6013c6ed9"
format_stamp: "Formatted at 2026-08-11 19:14:55 on dist-test-slave-t7g5"
server_key: "dc94c764599ca34a132358e493d1d1e8"
server_key_iv: "a0452383278c77200d518bf5b16dac6c"
server_key_version: "encryptionkey@0"
I20260811 19:14:55.501044 26439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/minicluster-data/master-1/wal/instance:
uuid: "581dcb09f4374097938f97b6013c6ed9"
format_stamp: "Formatted at 2026-08-11 19:14:55 on dist-test-slave-t7g5"
server_key: "dc94c764599ca34a132358e493d1d1e8"
server_key_iv: "a0452383278c77200d518bf5b16dac6c"
server_key_version: "encryptionkey@0"
I20260811 19:14:55.509042 26439 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.010s sys 0.000s
I20260811 19:14:55.514328 26462 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:14:55.516284 26439 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260811 19:14:55.516551 26439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/minicluster-data/master-1/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/minicluster-data/master-1/wal
uuid: "581dcb09f4374097938f97b6013c6ed9"
format_stamp: "Formatted at 2026-08-11 19:14:55 on dist-test-slave-t7g5"
server_key: "dc94c764599ca34a132358e493d1d1e8"
server_key_iv: "a0452383278c77200d518bf5b16dac6c"
server_key_version: "encryptionkey@0"
I20260811 19:14:55.516796 26439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:14:55.530433 26439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:14:55.535825 26439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:14:55.536183 26439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:14:55.558257 26439 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.189:43763
I20260811 19:14:55.558362 26547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.189:43763 every 8 connection(s)
I20260811 19:14:55.560200 26439 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/minicluster-data/master-1/data/info.pb
I20260811 19:14:55.564951 26551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:14:55.569639 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 26439
I20260811 19:14:55.569921 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/minicluster-data/master-1/wal/instance
I20260811 19:14:55.570495 3318 external_mini_cluster.cc:1428] Setting key f6beed4e73b68960390972ceb9fbfbc2
I20260811 19:14:55.573396 26551 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.61.190" port: 41645 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 19:14:55 dist-test-slave-t7g5 krb5kdc[26308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475695, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.189@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:14:55.600730 26553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:55.577632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36768 (local address 127.3.61.190:41645)
0811 19:14:55.578385 (+ 753us) server_negotiation.cc:206] Beginning negotiation
0811 19:14:55.578392 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:14:55.579208 (+ 816us) server_negotiation.cc:407] Connection header received
0811 19:14:55.582033 (+ 2825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:14:55.582038 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:14:55.582125 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:14:55.582232 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:14:55.584505 (+ 2273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:55.585357 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:55.587132 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:55.587727 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:55.587869 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:14:55.592189 (+ 4320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:14:55.592197 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:14:55.592201 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:14:55.592268 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:14:55.595965 (+ 3697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:14:55.597160 (+ 1195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:14:55.597166 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:14:55.597169 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:14:55.597342 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:14:55.598163 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:14:55.598168 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:14:55.598171 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:14:55.598630 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:14:55.598797 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 19:14:55.600030 (+ 1233us) server_negotiation.cc:299] Negotiation successful
0811 19:14:55.600322 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":408,"threads_started":1}
I20260811 19:14:55.601799 26554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:55.577969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:41645 (local address 127.0.0.1:36768)
0811 19:14:55.579003 (+ 1034us) negotiation.cc:107] Waiting for socket to connect
0811 19:14:55.579062 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 19:14:55.581556 (+ 2494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:14:55.582548 (+ 992us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:14:55.582560 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:14:55.583370 (+ 810us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:14:55.584242 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:14:55.584269 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:14:55.585553 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:14:55.585559 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:14:55.586957 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:14:55.586972 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:14:55.588028 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:14:55.588035 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:14:55.588250 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:14:55.589225 (+ 975us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:14:55.589267 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:14:55.591732 (+ 2465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:14:55.596210 (+ 4478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:14:55.596220 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:14:55.596236 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:14:55.596918 (+ 682us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:14:55.597561 (+ 643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:14:55.597567 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:14:55.597569 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:14:55.597996 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:14:55.598835 (+ 839us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:14:55.598844 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:14:55.599400 (+ 556us) client_negotiation.cc:769] Sending connection context
0811 19:14:55.599891 (+ 491us) client_negotiation.cc:240] Negotiation successful
0811 19:14:55.600481 (+ 590us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":712,"thread_start_us":438,"threads_started":1}
WARNING: no policy specified for kudu/127.3.61.188@KRBTEST.COM; defaulting to no policy
I20260811 19:14:55.603968 26551 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43763 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.188@KRBTEST.COM" created.
Aug 11 19:14:55 dist-test-slave-t7g5 krb5kdc[26308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475695, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.189@KRBTEST.COM for kudu/127.3.61.189@KRBTEST.COM
I20260811 19:14:55.623092 26554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:55.605042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.189:43763 (local address 127.0.0.1:57314)
0811 19:14:55.605314 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0811 19:14:55.605336 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 19:14:55.605452 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:14:55.607225 (+ 1773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:14:55.607234 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:14:55.607719 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:14:55.607919 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:14:55.607931 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:14:55.609815 (+ 1884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:14:55.609821 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:14:55.611014 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:14:55.611032 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:14:55.612074 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:14:55.612085 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:14:55.612266 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:14:55.612923 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:14:55.612945 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:14:55.615538 (+ 2593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:14:55.619286 (+ 3748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:14:55.619292 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:14:55.619296 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:14:55.619935 (+ 639us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:14:55.620738 (+ 803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:14:55.620745 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:14:55.620750 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:14:55.621004 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:14:55.622143 (+ 1139us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:14:55.622148 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:14:55.622294 (+ 146us) client_negotiation.cc:769] Sending connection context
0811 19:14:55.622449 (+ 155us) client_negotiation.cc:240] Negotiation successful
0811 19:14:55.622707 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91}
I20260811 19:14:55.623581 26556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:55.605190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57314 (local address 127.3.61.189:43763)
0811 19:14:55.606723 (+ 1533us) server_negotiation.cc:206] Beginning negotiation
0811 19:14:55.606729 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:14:55.606758 (+ 29us) server_negotiation.cc:407] Connection header received
0811 19:14:55.606817 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:14:55.606824 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:14:55.607040 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:14:55.607173 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:14:55.608208 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:55.609581 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:55.611216 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:55.611879 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:55.612006 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:14:55.615818 (+ 3812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:14:55.615836 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:14:55.615854 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:14:55.615887 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:14:55.618992 (+ 3105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:14:55.620147 (+ 1155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:14:55.620157 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:14:55.620160 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:14:55.620390 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:14:55.621169 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:14:55.621176 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:14:55.621179 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:14:55.621951 (+ 772us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:14:55.622224 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0811 19:14:55.622844 (+ 620us) server_negotiation.cc:299] Negotiation successful
0811 19:14:55.623161 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1359,"thread_start_us":836,"threads_started":1}
I20260811 19:14:55.626397 26551 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.61.188" port: 43463 } has no permanent_uuid. Determining permanent_uuid...
I20260811 19:14:55.627807 26554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:55.627216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:43463 (local address 127.0.0.1:50886)
0811 19:14:55.627440 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0811 19:14:55.627613 (+ 173us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.188:43463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
W20260811 19:14:55.628180 26469 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.61.188:43463: connect: Connection refused (error 111)
I20260811 19:14:55.630079 26554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:55.629574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:43463 (local address 127.0.0.1:50894)
0811 19:14:55.629814 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0811 19:14:55.629929 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.188:43463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
W20260811 19:14:55.630501 26551 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.61.188:43463: Network error: Client connection negotiation failed: client connection to 127.3.61.188:43463: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.188.keytab.
Entry for principal kudu/127.3.61.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.188.keytab.
I20260811 19:14:55.645880 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.188:43463
--webserver_interface=127.3.61.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.188.keytab
--principal=kudu/127.3.61.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:34969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.61.190:41645,127.3.61.189:43763,127.3.61.188:43463
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:14:55.668488 26429 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43763 } attempt: 5
Aug 11 19:14:55 dist-test-slave-t7g5 krb5kdc[26308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475694, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.190@KRBTEST.COM for kudu/127.3.61.189@KRBTEST.COM
I20260811 19:14:55.679972 26551 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.61.188" port: 43463 } attempt: 1
I20260811 19:14:55.681285 26554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:55.680680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:43463 (local address 127.0.0.1:50898)
0811 19:14:55.680927 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0811 19:14:55.681074 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.188:43463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260811 19:14:55.683094 26554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:55.682208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:43463 (local address 127.0.0.1:50900)
0811 19:14:55.682826 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0811 19:14:55.682922 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.188:43463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":474}
W20260811 19:14:55.683454 26551 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.61.188:43463: Network error: Client connection negotiation failed: client connection to 127.3.61.188:43463: connect: Connection refused (error 111)
I20260811 19:14:55.687073 26556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:55.669420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57320 (local address 127.3.61.189:43763)
0811 19:14:55.669589 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 19:14:55.669593 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:14:55.669619 (+ 26us) server_negotiation.cc:407] Connection header received
0811 19:14:55.669879 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:14:55.669884 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:14:55.669969 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:14:55.670047 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:14:55.671199 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:55.672493 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:55.674412 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:55.675001 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:55.675159 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:14:55.679414 (+ 4255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:14:55.679423 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:14:55.679428 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:14:55.679493 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:14:55.684017 (+ 4524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:14:55.684953 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:14:55.684958 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:14:55.684961 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:14:55.685103 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:14:55.685528 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:14:55.685531 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:14:55.685533 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:14:55.685935 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:14:55.686057 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 19:14:55.686640 (+ 583us) server_negotiation.cc:299] Negotiation successful
0811 19:14:55.686841 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 19:14:55.687172 26431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:55.669282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.189:43763 (local address 127.0.0.1:57320)
0811 19:14:55.669537 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0811 19:14:55.669559 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 19:14:55.669733 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:14:55.670123 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:14:55.670128 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:14:55.670670 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:14:55.670937 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:14:55.670951 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:14:55.672691 (+ 1740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:14:55.672698 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:14:55.674165 (+ 1467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:14:55.674184 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:14:55.675220 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:14:55.675226 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:14:55.675427 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:14:55.676278 (+ 851us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:14:55.676304 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:14:55.679126 (+ 2822us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:14:55.684189 (+ 5063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:14:55.684195 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:14:55.684199 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:14:55.684761 (+ 562us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:14:55.685219 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:14:55.685224 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:14:55.685228 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:14:55.685404 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:14:55.686093 (+ 689us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:14:55.686100 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:14:55.686304 (+ 204us) client_negotiation.cc:769] Sending connection context
0811 19:14:55.686508 (+ 204us) client_negotiation.cc:240] Negotiation successful
0811 19:14:55.686828 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86}
I20260811 19:14:55.688786 26429 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.61.188" port: 43463 } has no permanent_uuid. Determining permanent_uuid...
I20260811 19:14:55.690322 26431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:55.689752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:43463 (local address 127.0.0.1:50906)
0811 19:14:55.690005 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0811 19:14:55.690124 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.188:43463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260811 19:14:55.692077 26431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:55.691603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:43463 (local address 127.0.0.1:50912)
0811 19:14:55.691806 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0811 19:14:55.691922 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.188:43463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260811 19:14:55.692312 26429 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.61.188:43463: Network error: Client connection negotiation failed: client connection to 127.3.61.188:43463: connect: Connection refused (error 111)
I20260811 19:14:55.743570 26429 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.61.188" port: 43463 } attempt: 1
I20260811 19:14:55.744781 26431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:55.744311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:43463 (local address 127.0.0.1:50916)
0811 19:14:55.744513 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 19:14:55.744613 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.188:43463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260811 19:14:55.745985 26431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:55.745566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:43463 (local address 127.0.0.1:50928)
0811 19:14:55.745777 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 19:14:55.745857 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.188:43463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
W20260811 19:14:55.746217 26429 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.61.188:43463: Network error: Client connection negotiation failed: client connection to 127.3.61.188:43463: connect: Connection refused (error 111)
I20260811 19:14:55.751719 26551 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.61.188" port: 43463 } attempt: 2
I20260811 19:14:55.753371 26554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:55.752725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:43463 (local address 127.0.0.1:50938)
0811 19:14:55.753009 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0811 19:14:55.753153 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.188:43463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260811 19:14:55.754745 26554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:55.754288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:43463 (local address 127.0.0.1:50942)
0811 19:14:55.754467 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 19:14:55.754571 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.188:43463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260811 19:14:55.755048 26551 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.61.188:43463: Network error: Client connection negotiation failed: client connection to 127.3.61.188:43463: connect: Connection refused (error 111)
I20260811 19:14:55.814489 26429 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.61.188" port: 43463 } attempt: 2
I20260811 19:14:55.815858 26431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:55.815202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:43463 (local address 127.0.0.1:50954)
0811 19:14:55.815457 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0811 19:14:55.815563 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.188:43463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260811 19:14:55.817380 26431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:55.816665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:43463 (local address 127.0.0.1:50960)
0811 19:14:55.817023 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0811 19:14:55.817175 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.188:43463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160}
W20260811 19:14:55.817734 26429 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.61.188:43463: Network error: Client connection negotiation failed: client connection to 127.3.61.188:43463: connect: Connection refused (error 111)
I20260811 19:14:55.846339 26551 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.61.188" port: 43463 } attempt: 3
I20260811 19:14:55.847991 26554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:55.847290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:43463 (local address 127.0.0.1:50968)
0811 19:14:55.847619 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0811 19:14:55.847764 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.188:43463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260811 19:14:55.849771 26554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:55.848897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:43463 (local address 127.0.0.1:50972)
0811 19:14:55.849267 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0811 19:14:55.849405 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.188:43463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217}
W20260811 19:14:55.850099 26551 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.61.188:43463: Network error: Client connection negotiation failed: client connection to 127.3.61.188:43463: connect: Connection refused (error 111)
I20260811 19:14:55.923995 26429 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.61.188" port: 43463 } attempt: 3
I20260811 19:14:55.925299 26431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:55.924797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:43463 (local address 127.0.0.1:50984)
0811 19:14:55.925016 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 19:14:55.925116 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.188:43463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260811 19:14:55.926414 26431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:55.926028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:43463 (local address 127.0.0.1:50986)
0811 19:14:55.926228 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0811 19:14:55.926306 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.188:43463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
W20260811 19:14:55.926642 26429 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.61.188:43463: Network error: Client connection negotiation failed: client connection to 127.3.61.188:43463: connect: Connection refused (error 111)
I20260811 19:14:55.993373 26551 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.61.188" port: 43463 } attempt: 4
I20260811 19:14:55.995038 26554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:55.994317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:43463 (local address 127.0.0.1:50990)
0811 19:14:55.994669 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0811 19:14:55.994820 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.188:43463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260811 19:14:55.996438 26554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:55.995973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:43463 (local address 127.0.0.1:50996)
0811 19:14:55.996161 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0811 19:14:55.996281 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.188:43463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260811 19:14:55.996773 26551 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.61.188:43463: Network error: Client connection negotiation failed: client connection to 127.3.61.188:43463: connect: Connection refused (error 111)
W20260811 19:14:56.033139 26560 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:14:56.033700 26560 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:14:56.033855 26560 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:14:56.042814 26560 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:14:56.043008 26560 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:14:56.043064 26560 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:14:56.043133 26560 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:14:56.043175 26560 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:14:56.055066 26560 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:34969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.3.61.190:41645,127.3.61.189:43763,127.3.61.188:43463
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.188.keytab
--principal=kudu/127.3.61.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.188:43463
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.3.61.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:14:56.056550 26560 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:14:56.057981 26560 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:14:56.073256 26566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:14:56.073271 26569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:14:56.073518 26567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:14:56.075246 26560 server_base.cc:1034] running on GCE node
I20260811 19:14:56.076165 26560 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:14:56.077847 26560 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:14:56.079102 26560 hybrid_clock.cc:648] HybridClock initialized: now 1786475696079042 us; error 41 us; skew 500 ppm
I20260811 19:14:56.079525 26560 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:14:56 dist-test-slave-t7g5 krb5kdc[26308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475696, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:14:56.082978 26560 init.cc:377] Logged in from keytab as kudu/127.3.61.188@KRBTEST.COM (short username kudu)
I20260811 19:14:56.084722 26560 webserver.cc:460] Webserver started at http://127.3.61.188:41391/ using document root <none> and password file <none>
I20260811 19:14:56.085491 26560 fs_manager.cc:359] Metadata directory not provided
I20260811 19:14:56.085605 26560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:14:56.085965 26560 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:14:56.089177 26560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/minicluster-data/master-2/data/instance:
uuid: "d72d675a81024955a8aa1324aae4782f"
format_stamp: "Formatted at 2026-08-11 19:14:56 on dist-test-slave-t7g5"
server_key: "9321061ebb2c9c1e9a43074c9d064f8e"
server_key_iv: "e314afe85cf4e8a108e018b418768e17"
server_key_version: "encryptionkey@0"
I20260811 19:14:56.090124 26560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/minicluster-data/master-2/wal/instance:
uuid: "d72d675a81024955a8aa1324aae4782f"
format_stamp: "Formatted at 2026-08-11 19:14:56 on dist-test-slave-t7g5"
server_key: "9321061ebb2c9c1e9a43074c9d064f8e"
server_key_iv: "e314afe85cf4e8a108e018b418768e17"
server_key_version: "encryptionkey@0"
I20260811 19:14:56.096124 26560 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 19:14:56.100852 26580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:14:56.102533 26560 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 19:14:56.102753 26560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/minicluster-data/master-2/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/minicluster-data/master-2/wal
uuid: "d72d675a81024955a8aa1324aae4782f"
format_stamp: "Formatted at 2026-08-11 19:14:56 on dist-test-slave-t7g5"
server_key: "9321061ebb2c9c1e9a43074c9d064f8e"
server_key_iv: "e314afe85cf4e8a108e018b418768e17"
server_key_version: "encryptionkey@0"
I20260811 19:14:56.103039 26560 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:14:56.103858 26429 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.61.188" port: 43463 } attempt: 4
I20260811 19:14:56.105209 26431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:56.104690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:43463 (local address 127.0.0.1:51008)
0811 19:14:56.104953 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0811 19:14:56.105063 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.188:43463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260811 19:14:56.106552 26431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:56.106124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:43463 (local address 127.0.0.1:51020)
0811 19:14:56.106308 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 19:14:56.106397 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.188:43463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260811 19:14:56.106822 26429 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.61.188:43463: Network error: Client connection negotiation failed: client connection to 127.3.61.188:43463: connect: Connection refused (error 111)
I20260811 19:14:56.127964 26560 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:14:56.132635 26560 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:14:56.132910 26560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:14:56.154274 26560 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.188:43463
I20260811 19:14:56.154404 26657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.188:43463 every 8 connection(s)
I20260811 19:14:56.156256 26560 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/minicluster-data/master-2/data/info.pb
I20260811 19:14:56.161095 26658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:14:56.161779 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 26560
I20260811 19:14:56.162022 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786475454804547-3318-0/minicluster-data/master-2/wal/instance
I20260811 19:14:56.162868 3318 external_mini_cluster.cc:1428] Setting key b90b2c349106b634b0692d66b72c65a4
I20260811 19:14:56.172923 26658 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.61.190" port: 41645 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 19:14:56 dist-test-slave-t7g5 krb5kdc[26308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.188@KRBTEST.COM
Aug 11 19:14:56 dist-test-slave-t7g5 krb5kdc[26308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475696, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.188@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:14:56.191752 26661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:56.165223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51034 (local address 127.3.61.188:43463)
0811 19:14:56.166196 (+ 973us) server_negotiation.cc:206] Beginning negotiation
0811 19:14:56.166219 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 19:14:56.166270 (+ 51us) server_negotiation.cc:407] Connection header received
0811 19:14:56.169106 (+ 2836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:14:56.169215 (+ 109us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:14:56.169873 (+ 658us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:14:56.170355 (+ 482us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:14:56.172112 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:56.173513 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:56.175055 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:56.175832 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:56.176031 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:14:56.181130 (+ 5099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:14:56.181138 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:14:56.181151 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:14:56.181194 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:14:56.184996 (+ 3802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:14:56.186111 (+ 1115us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:14:56.186121 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:14:56.186131 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:14:56.186364 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:14:56.186968 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:14:56.186977 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:14:56.186981 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:14:56.188059 (+ 1078us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:14:56.188335 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0811 19:14:56.189350 (+ 1015us) server_negotiation.cc:299] Negotiation successful
0811 19:14:56.190172 (+ 822us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":735,"thread_start_us":457,"threads_started":1}
I20260811 19:14:56.195139 26664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:56.176481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:41645 (local address 127.0.0.1:36774)
0811 19:14:56.177589 (+ 1108us) negotiation.cc:107] Waiting for socket to connect
0811 19:14:56.177677 (+ 88us) client_negotiation.cc:174] Beginning negotiation
0811 19:14:56.177872 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:14:56.179035 (+ 1163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:14:56.179044 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:14:56.180161 (+ 1117us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:14:56.180575 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:14:56.180592 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:14:56.181774 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:14:56.181778 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:14:56.183445 (+ 1667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:14:56.183462 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:14:56.184283 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:14:56.184291 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:14:56.184492 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:14:56.185265 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:14:56.185282 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:14:56.187774 (+ 2492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:14:56.191807 (+ 4033us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:14:56.191821 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:14:56.191829 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:14:56.192521 (+ 692us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:14:56.192997 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:14:56.193002 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:14:56.193006 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:14:56.193204 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:14:56.193923 (+ 719us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:14:56.193931 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:14:56.194167 (+ 236us) client_negotiation.cc:769] Sending connection context
0811 19:14:56.194385 (+ 218us) client_negotiation.cc:240] Negotiation successful
0811 19:14:56.194713 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":765,"thread_start_us":521,"threads_started":1}
I20260811 19:14:56.195497 26662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:56.176490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36774 (local address 127.3.61.190:41645)
0811 19:14:56.178459 (+ 1969us) server_negotiation.cc:206] Beginning negotiation
0811 19:14:56.178464 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:14:56.178488 (+ 24us) server_negotiation.cc:407] Connection header received
0811 19:14:56.178672 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:14:56.178678 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:14:56.178764 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:14:56.178901 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:14:56.180788 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:56.181597 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:56.183626 (+ 2029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:56.184061 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:56.184171 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:14:56.187991 (+ 3820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:14:56.187997 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:14:56.188000 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:14:56.188035 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:14:56.191629 (+ 3594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:14:56.192660 (+ 1031us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:14:56.192666 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:14:56.192670 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:14:56.192850 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:14:56.193342 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:14:56.193347 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:14:56.193350 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:14:56.193764 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:14:56.193933 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 19:14:56.194528 (+ 595us) server_negotiation.cc:299] Negotiation successful
0811 19:14:56.194779 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1838,"thread_start_us":1665,"threads_started":1}
I20260811 19:14:56.197237 26658 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43763 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 19:14:56 dist-test-slave-t7g5 krb5kdc[26308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475696, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.188@KRBTEST.COM for kudu/127.3.61.189@KRBTEST.COM
Aug 11 19:14:56 dist-test-slave-t7g5 krb5kdc[26308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:14:56.216953 26664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:56.198075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.189:43763 (local address 127.0.0.1:57322)
0811 19:14:56.198342 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0811 19:14:56.198359 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 19:14:56.198455 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:14:56.199397 (+ 942us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:14:56.199401 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:14:56.199755 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:14:56.199916 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:14:56.199925 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:14:56.201625 (+ 1700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:14:56.201633 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:14:56.203417 (+ 1784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:14:56.203442 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:14:56.204873 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:14:56.204881 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:14:56.205165 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:14:56.206113 (+ 948us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:14:56.206150 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:14:56.209430 (+ 3280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:14:56.213857 (+ 4427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:14:56.213866 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:14:56.213871 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:14:56.214512 (+ 641us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:14:56.215052 (+ 540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:14:56.215055 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:14:56.215058 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:14:56.215226 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:14:56.215842 (+ 616us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:14:56.215848 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:14:56.216036 (+ 188us) client_negotiation.cc:769] Sending connection context
0811 19:14:56.216168 (+ 132us) client_negotiation.cc:240] Negotiation successful
0811 19:14:56.216546 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171}
I20260811 19:14:56.217046 26665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:56.198290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57322 (local address 127.3.61.189:43763)
0811 19:14:56.199057 (+ 767us) server_negotiation.cc:206] Beginning negotiation
0811 19:14:56.199064 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:14:56.199085 (+ 21us) server_negotiation.cc:407] Connection header received
0811 19:14:56.199146 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:14:56.199151 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:14:56.199236 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:14:56.199355 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:14:56.200212 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:56.201343 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:56.203645 (+ 2302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:56.204580 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:56.204740 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:14:56.209777 (+ 5037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:14:56.209785 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:14:56.209788 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:14:56.209831 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:14:56.213566 (+ 3735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:14:56.214714 (+ 1148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:14:56.214721 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:14:56.214723 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:14:56.214920 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:14:56.215350 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:14:56.215354 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:14:56.215356 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:14:56.215720 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:14:56.215873 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 19:14:56.216303 (+ 430us) server_negotiation.cc:299] Negotiation successful
0811 19:14:56.216596 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":648,"thread_start_us":519,"threads_started":1}
I20260811 19:14:56.217937 26662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:56.197071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36776 (local address 127.3.61.190:41645)
0811 19:14:56.197646 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0811 19:14:56.197682 (+ 36us) server_negotiation.cc:399] Waiting for connection header
0811 19:14:56.197699 (+ 17us) server_negotiation.cc:407] Connection header received
0811 19:14:56.197757 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:14:56.197762 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:14:56.197853 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:14:56.197964 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:14:56.199832 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:56.201254 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:56.203768 (+ 2514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:56.204600 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:56.204740 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:14:56.210836 (+ 6096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:14:56.210846 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:14:56.210852 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:14:56.210922 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:14:56.214437 (+ 3515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:14:56.215269 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:14:56.215275 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:14:56.215279 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:14:56.215427 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:14:56.215838 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:14:56.215843 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:14:56.215847 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:14:56.216158 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:14:56.216453 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0811 19:14:56.216742 (+ 289us) server_negotiation.cc:299] Negotiation successful
0811 19:14:56.216992 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459}
I20260811 19:14:56.218470 26658 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.61.188" port: 43463 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 19:14:56 dist-test-slave-t7g5 krb5kdc[26308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475696, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.188@KRBTEST.COM for kudu/127.3.61.188@KRBTEST.COM
Aug 11 19:14:56 dist-test-slave-t7g5 krb5kdc[26308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.189@KRBTEST.COM
I20260811 19:14:56.235668 26664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:56.219427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:43463 (local address 127.0.0.1:51036)
0811 19:14:56.219610 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 19:14:56.219631 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 19:14:56.219741 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:14:56.220319 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:14:56.220325 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:14:56.220832 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:14:56.221168 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:14:56.221185 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:14:56.222642 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:14:56.222647 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:14:56.224135 (+ 1488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:14:56.224152 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:14:56.225204 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:14:56.225209 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:14:56.225360 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:14:56.226113 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:14:56.226137 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:14:56.229194 (+ 3057us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:14:56.232464 (+ 3270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:14:56.232471 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:14:56.232475 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:14:56.233109 (+ 634us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:14:56.233637 (+ 528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:14:56.233643 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:14:56.233647 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:14:56.233890 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:14:56.234485 (+ 595us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:14:56.234491 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:14:56.234659 (+ 168us) client_negotiation.cc:769] Sending connection context
0811 19:14:56.234846 (+ 187us) client_negotiation.cc:240] Negotiation successful
0811 19:14:56.235219 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 19:14:56.235736 26661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:56.219700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51036 (local address 127.3.61.188:43463)
0811 19:14:56.219950 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0811 19:14:56.219958 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:14:56.219981 (+ 23us) server_negotiation.cc:407] Connection header received
0811 19:14:56.220048 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:14:56.220053 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:14:56.220160 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:14:56.220294 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:14:56.221346 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:56.222472 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:56.224307 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:56.224999 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:56.225144 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:14:56.229437 (+ 4293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:14:56.229445 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:14:56.229450 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:14:56.229491 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:14:56.232234 (+ 2743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:14:56.233291 (+ 1057us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:14:56.233295 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:14:56.233298 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:14:56.233473 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:14:56.234021 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:14:56.234025 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:14:56.234027 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:14:56.234359 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:14:56.234476 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 19:14:56.234967 (+ 491us) server_negotiation.cc:299] Negotiation successful
0811 19:14:56.235374 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":178,"server-negotiator.queue_time_us":79}
I20260811 19:14:56.238437 26665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:56.220133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57328 (local address 127.3.61.189:43763)
0811 19:14:56.220289 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 19:14:56.220294 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:14:56.220562 (+ 268us) server_negotiation.cc:407] Connection header received
0811 19:14:56.220686 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:14:56.220691 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:14:56.220773 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:14:56.220882 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:14:56.222826 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:56.224034 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:56.225760 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:56.226267 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:56.226380 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:14:56.231577 (+ 5197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:14:56.231584 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:14:56.231588 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:14:56.231629 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:14:56.235385 (+ 3756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:14:56.236292 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:14:56.236298 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:14:56.236300 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:14:56.236473 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:14:56.236870 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:14:56.236874 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:14:56.236876 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:14:56.237253 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:14:56.237527 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0811 19:14:56.237851 (+ 324us) server_negotiation.cc:299] Negotiation successful
0811 19:14:56.238099 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 19:14:56.242672 26658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d72d675a81024955a8aa1324aae4782f: Bootstrap starting.
I20260811 19:14:56.245069 3318 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:14:56.246620 26658 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d72d675a81024955a8aa1324aae4782f: Neither blocks nor log segments found. Creating new log.
I20260811 19:14:56.247711 26658 log.cc:824] T 00000000000000000000000000000000 P d72d675a81024955a8aa1324aae4782f: Log is configured to *not* fsync() on all Append() calls
I20260811 19:14:56.251497 26658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d72d675a81024955a8aa1324aae4782f: No bootstrap required, opened a new log
I20260811 19:14:56.262298 26658 raft_consensus.cc:348] T 00000000000000000000000000000000 P d72d675a81024955a8aa1324aae4782f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ad5c7bfe7a14c3fb4408d6ed8a11057" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 41645 } } peers { permanent_uuid: "581dcb09f4374097938f97b6013c6ed9" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43763 } } peers { permanent_uuid: "d72d675a81024955a8aa1324aae4782f" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 43463 } }
I20260811 19:14:56.263301 26658 raft_consensus.cc:374] T 00000000000000000000000000000000 P d72d675a81024955a8aa1324aae4782f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:14:56.263623 26658 raft_consensus.cc:729] T 00000000000000000000000000000000 P d72d675a81024955a8aa1324aae4782f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72d675a81024955a8aa1324aae4782f, State: Initialized, Role: FOLLOWER
I20260811 19:14:56.264838 26658 consensus_queue.cc:260] T 00000000000000000000000000000000 P d72d675a81024955a8aa1324aae4782f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ad5c7bfe7a14c3fb4408d6ed8a11057" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 41645 } } peers { permanent_uuid: "581dcb09f4374097938f97b6013c6ed9" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43763 } } peers { permanent_uuid: "d72d675a81024955a8aa1324aae4782f" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 43463 } }
I20260811 19:14:56.274868 26661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:56.252237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51042 (local address 127.3.61.188:43463)
0811 19:14:56.252469 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 19:14:56.252476 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:14:56.254788 (+ 2312us) server_negotiation.cc:407] Connection header received
0811 19:14:56.254979 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:14:56.254989 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:14:56.255091 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:14:56.255202 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:14:56.258124 (+ 2922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:56.259309 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:56.261604 (+ 2295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:56.262371 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:56.262470 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:14:56.266958 (+ 4488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:14:56.266965 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:14:56.266970 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:14:56.267015 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:14:56.270689 (+ 3674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:14:56.271595 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:14:56.271602 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:14:56.271605 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:14:56.271795 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:14:56.272277 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:14:56.272282 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:14:56.272285 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:14:56.272709 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:14:56.272979 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0811 19:14:56.274198 (+ 1219us) server_negotiation.cc:299] Negotiation successful
0811 19:14:56.274442 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 19:14:56.275772 26665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:56.252018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57338 (local address 127.3.61.189:43763)
0811 19:14:56.252226 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 19:14:56.252233 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:14:56.254268 (+ 2035us) server_negotiation.cc:407] Connection header received
0811 19:14:56.255105 (+ 837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:14:56.255111 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:14:56.255199 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:14:56.255287 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:14:56.258505 (+ 3218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:56.259654 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:56.262118 (+ 2464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:56.262760 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:56.262852 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:14:56.267989 (+ 5137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:14:56.267997 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:14:56.268002 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:14:56.268050 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:14:56.271849 (+ 3799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:14:56.272771 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:14:56.272777 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:14:56.272781 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:14:56.272936 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:14:56.273870 (+ 934us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:14:56.273876 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:14:56.273879 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:14:56.274275 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:14:56.274506 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0811 19:14:56.275142 (+ 636us) server_negotiation.cc:299] Negotiation successful
0811 19:14:56.275397 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 19:14:56.278059 26662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:56.252498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36788 (local address 127.3.61.190:41645)
0811 19:14:56.252673 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 19:14:56.252679 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:14:56.258345 (+ 5666us) server_negotiation.cc:407] Connection header received
0811 19:14:56.258523 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:14:56.258530 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:14:56.258615 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:14:56.258684 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:14:56.261170 (+ 2486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:56.262204 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:56.265520 (+ 3316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:56.266298 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:56.266426 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:14:56.270365 (+ 3939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:14:56.270374 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:14:56.270380 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:14:56.270537 (+ 157us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:14:56.274336 (+ 3799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:14:56.275374 (+ 1038us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:14:56.275381 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:14:56.275386 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:14:56.275601 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:14:56.276094 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:14:56.276100 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:14:56.276105 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:14:56.276526 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:14:56.276786 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0811 19:14:56.277426 (+ 640us) server_negotiation.cc:299] Negotiation successful
0811 19:14:56.277708 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 19:14:56.277671 26682 sys_catalog.cc:455] T 00000000000000000000000000000000 P d72d675a81024955a8aa1324aae4782f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ad5c7bfe7a14c3fb4408d6ed8a11057" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 41645 } } peers { permanent_uuid: "581dcb09f4374097938f97b6013c6ed9" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43763 } } peers { permanent_uuid: "d72d675a81024955a8aa1324aae4782f" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 43463 } } }
I20260811 19:14:56.278196 26682 sys_catalog.cc:458] T 00000000000000000000000000000000 P d72d675a81024955a8aa1324aae4782f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:14:56.279837 26658 sys_catalog.cc:564] T 00000000000000000000000000000000 P d72d675a81024955a8aa1324aae4782f [sys.catalog]: configured and running, proceeding with master startup.
W20260811 19:14:56.292006 26697 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d72d675a81024955a8aa1324aae4782f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 19:14:56.292176 26697 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 19:14:56.295987 26551 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.61.188" port: 43463 } attempt: 5
I20260811 19:14:56.289587 26560 master_runner.cc:186] Error getting master registration for 127.3.61.190:41645: OK, instance_id { permanent_uuid: "7ad5c7bfe7a14c3fb4408d6ed8a11057" instance_seqno: 1786475694943074 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 19:14:56.298529 26560 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 11 19:14:56 dist-test-slave-t7g5 krb5kdc[26308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475695, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.189@KRBTEST.COM for kudu/127.3.61.188@KRBTEST.COM
I20260811 19:14:56.316833 26661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:56.297462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51052 (local address 127.3.61.188:43463)
0811 19:14:56.297695 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 19:14:56.297702 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:14:56.297725 (+ 23us) server_negotiation.cc:407] Connection header received
0811 19:14:56.297786 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:14:56.297791 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:14:56.297876 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:14:56.297977 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:14:56.299940 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:56.301395 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:56.303264 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:56.304124 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:56.304248 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:14:56.308960 (+ 4712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:14:56.308966 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:14:56.308969 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:14:56.309025 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:14:56.312290 (+ 3265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:14:56.313323 (+ 1033us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:14:56.313341 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:14:56.313344 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:14:56.313524 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:14:56.314070 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:14:56.314077 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:14:56.314080 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:14:56.314461 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:14:56.314634 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 19:14:56.316114 (+ 1480us) server_negotiation.cc:299] Negotiation successful
0811 19:14:56.316429 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 19:14:56.317548 26554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:56.296708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:43463 (local address 127.0.0.1:51052)
0811 19:14:56.297085 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0811 19:14:56.297105 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 19:14:56.297202 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:14:56.298295 (+ 1093us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:14:56.298301 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:14:56.299143 (+ 842us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:14:56.299537 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:14:56.299558 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:14:56.301569 (+ 2011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:14:56.301575 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:14:56.303020 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:14:56.303039 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:14:56.304305 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:14:56.304312 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:14:56.304572 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:14:56.305733 (+ 1161us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:14:56.305766 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:14:56.308733 (+ 2967us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:14:56.312480 (+ 3747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:14:56.312488 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:14:56.312492 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:14:56.313128 (+ 636us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:14:56.313682 (+ 554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:14:56.313689 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:14:56.313693 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:14:56.313889 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:14:56.314602 (+ 713us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:14:56.314607 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:14:56.314811 (+ 204us) client_negotiation.cc:769] Sending connection context
0811 19:14:56.314965 (+ 154us) client_negotiation.cc:240] Negotiation successful
0811 19:14:56.317153 (+ 2188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251}
I20260811 19:14:56.324404 26551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 581dcb09f4374097938f97b6013c6ed9: Bootstrap starting.
I20260811 19:14:56.329047 26551 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 581dcb09f4374097938f97b6013c6ed9: Neither blocks nor log segments found. Creating new log.
I20260811 19:14:56.330685 26551 log.cc:824] T 00000000000000000000000000000000 P 581dcb09f4374097938f97b6013c6ed9: Log is configured to *not* fsync() on all Append() calls
I20260811 19:14:56.335045 26551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 581dcb09f4374097938f97b6013c6ed9: No bootstrap required, opened a new log
I20260811 19:14:56.346771 26551 raft_consensus.cc:348] T 00000000000000000000000000000000 P 581dcb09f4374097938f97b6013c6ed9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ad5c7bfe7a14c3fb4408d6ed8a11057" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 41645 } } peers { permanent_uuid: "581dcb09f4374097938f97b6013c6ed9" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43763 } } peers { permanent_uuid: "d72d675a81024955a8aa1324aae4782f" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 43463 } }
I20260811 19:14:56.347394 26551 raft_consensus.cc:374] T 00000000000000000000000000000000 P 581dcb09f4374097938f97b6013c6ed9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:14:56.347544 26551 raft_consensus.cc:729] T 00000000000000000000000000000000 P 581dcb09f4374097938f97b6013c6ed9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 581dcb09f4374097938f97b6013c6ed9, State: Initialized, Role: FOLLOWER
I20260811 19:14:56.348467 26551 consensus_queue.cc:260] T 00000000000000000000000000000000 P 581dcb09f4374097938f97b6013c6ed9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ad5c7bfe7a14c3fb4408d6ed8a11057" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 41645 } } peers { permanent_uuid: "581dcb09f4374097938f97b6013c6ed9" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43763 } } peers { permanent_uuid: "d72d675a81024955a8aa1324aae4782f" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 43463 } }
I20260811 19:14:56.349794 26702 sys_catalog.cc:455] T 00000000000000000000000000000000 P 581dcb09f4374097938f97b6013c6ed9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ad5c7bfe7a14c3fb4408d6ed8a11057" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 41645 } } peers { permanent_uuid: "581dcb09f4374097938f97b6013c6ed9" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43763 } } peers { permanent_uuid: "d72d675a81024955a8aa1324aae4782f" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 43463 } } }
I20260811 19:14:56.350369 26702 sys_catalog.cc:458] T 00000000000000000000000000000000 P 581dcb09f4374097938f97b6013c6ed9 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:14:56.351281 26551 sys_catalog.cc:564] T 00000000000000000000000000000000 P 581dcb09f4374097938f97b6013c6ed9 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 19:14:56.368336 26718 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 581dcb09f4374097938f97b6013c6ed9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 19:14:56.368626 26718 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 19:14:56.359990 26439 master_runner.cc:186] Error getting master registration for 127.3.61.190:41645: OK, instance_id { permanent_uuid: "7ad5c7bfe7a14c3fb4408d6ed8a11057" instance_seqno: 1786475694943074 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 19:14:56.371790 26439 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:14:56.384048 26429 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.61.188" port: 43463 } attempt: 5
W20260811 19:14:56.385254 26587 tablet.cc:2369] T 00000000000000000000000000000000 P d72d675a81024955a8aa1324aae4782f: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 19:14:56 dist-test-slave-t7g5 krb5kdc[26308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475694, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.190@KRBTEST.COM for kudu/127.3.61.188@KRBTEST.COM
I20260811 19:14:56.402899 26431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:56.384821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:43463 (local address 127.0.0.1:51068)
0811 19:14:56.385123 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0811 19:14:56.385145 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 19:14:56.385271 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:14:56.385727 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:14:56.385732 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:14:56.386277 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:14:56.386511 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:14:56.386525 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:14:56.388029 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:14:56.388035 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:14:56.389572 (+ 1537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:14:56.389592 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:14:56.390716 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:14:56.390723 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:14:56.390909 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:14:56.391800 (+ 891us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:14:56.391837 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:14:56.395033 (+ 3196us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:14:56.399750 (+ 4717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:14:56.399756 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:14:56.399761 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:14:56.400348 (+ 587us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:14:56.400827 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:14:56.400832 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:14:56.400836 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:14:56.401044 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:14:56.401748 (+ 704us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:14:56.401754 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:14:56.401949 (+ 195us) client_negotiation.cc:769] Sending connection context
0811 19:14:56.402107 (+ 158us) client_negotiation.cc:240] Negotiation successful
0811 19:14:56.402430 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170}
I20260811 19:14:56.403141 26661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:14:56.385042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51068 (local address 127.3.61.188:43463)
0811 19:14:56.385235 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 19:14:56.385241 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:14:56.385265 (+ 24us) server_negotiation.cc:407] Connection header received
0811 19:14:56.385422 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:14:56.385427 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:14:56.385508 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:14:56.385622 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:14:56.386689 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:56.387858 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:56.389784 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:14:56.390525 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:14:56.390678 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:14:56.395296 (+ 4618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:14:56.395305 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:14:56.395310 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:14:56.395359 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:14:56.399537 (+ 4178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:14:56.400508 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:14:56.400514 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:14:56.400518 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:14:56.400685 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:14:56.401177 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:14:56.401182 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:14:56.401186 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:14:56.401583 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:14:56.401798 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0811 19:14:56.402269 (+ 471us) server_negotiation.cc:299] Negotiation successful
0811 19:14:56.402650 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 19:14:56.410544 26429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ad5c7bfe7a14c3fb4408d6ed8a11057: Bootstrap starting.
I20260811 19:14:56.415565 26429 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ad5c7bfe7a14c3fb4408d6ed8a11057: Neither blocks nor log segments found. Creating new log.
I20260811 19:14:56.417165 26429 log.cc:824] T 00000000000000000000000000000000 P 7ad5c7bfe7a14c3fb4408d6ed8a11057: Log is configured to *not* fsync() on all Append() calls
I20260811 19:14:56.422535 26429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ad5c7bfe7a14c3fb4408d6ed8a11057: No bootstrap required, opened a new log
I20260811 19:14:56.424082 26702 raft_consensus.cc:482] T 00000000000000000000000000000000 P 581dcb09f4374097938f97b6013c6ed9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:14:56.424412 26702 raft_consensus.cc:504] T 00000000000000000000000000000000 P 581dcb09f4374097938f97b6013c6ed9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ad5c7bfe7a14c3fb4408d6ed8a11057" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 41645 } } peers { permanent_uuid: "581dcb09f4374097938f97b6013c6ed9" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43763 } } peers { permanent_uuid: "d72d675a81024955a8aa1324aae4782f" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 43463 } }
I20260811 19:14:56.426326 26702 leader_election.cc:288] T 00000000000000000000000000000000 P 581dcb09f4374097938f97b6013c6ed9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ad5c7bfe7a14c3fb4408d6ed8a11057 (127.3.61.190:41645), d72d675a81024955a8aa1324aae4782f (127.3.61.188:43463)
I20260811 19:14:56.427511 26620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "581dcb09f4374097938f97b6013c6ed9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72d675a81024955a8aa1324aae4782f" is_pre_election: true
I20260811 19:14:56.428194 26620 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d72d675a81024955a8aa1324aae4782f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 581dcb09f4374097938f97b6013c6ed9 in term 0.
I20260811 19:14:56.428964 26469 leader_election.cc:302] T 00000000000000000000000000000000 P 581dcb09f4374097938f97b6013c6ed9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 581dcb09f4374097938f97b6013c6ed9, d72d675a81024955a8aa1324aae4782f; no voters:
I20260811 19:14:56.429859 26702 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 581dcb09f4374097938f97b6013c6ed9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:14:56.430037 26702 raft_consensus.cc:482] T 00000000000000000000000000000000 P 581dcb09f4374097938f97b6013c6ed9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:14:56.430150 26702 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 581dcb09f4374097938f97b6013c6ed9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:14:56.439095 26702 raft_consensus.cc:504] T 00000000000000000000000000000000 P 581dcb09f4374097938f97b6013c6ed9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ad5c7bfe7a14c3fb4408d6ed8a11057" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 41645 } } peers { permanent_uuid: "581dcb09f4374097938f97b6013c6ed9" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43763 } } peers { permanent_uuid: "d72d675a81024955a8aa1324aae4782f" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 43463 } }
I20260811 19:14:56.439401 26429 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ad5c7bfe7a14c3fb4408d6ed8a11057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ad5c7bfe7a14c3fb4408d6ed8a11057" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 41645 } } peers { permanent_uuid: "581dcb09f4374097938f97b6013c6ed9" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43763 } } peers { permanent_uuid: "d72d675a81024955a8aa1324aae4782f" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 43463 } }
I20260811 19:14:56.439949 26429 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7ad5c7bfe7a14c3fb4408d6ed8a11057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:14:56.440124 26429 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ad5c7bfe7a14c3fb4408d6ed8a11057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ad5c7bfe7a14c3fb4408d6ed8a11057, State: Initialized, Role: FOLLOWER
I20260811 19:14:56.441071 26429 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ad5c7bfe7a14c3fb4408d6ed8a11057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ad5c7bfe7a14c3fb4408d6ed8a11057" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 41645 } } peers { permanent_uuid: "581dcb09f4374097938f97b6013c6ed9" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43763 } } peers { permanent_uuid: "d72d675a81024955a8aa1324aae4782f" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 43463 } }
I20260811 19:14:56.443763 26702 leader_election.cc:288] T 00000000000000000000000000000000 P 581dcb09f4374097938f97b6013c6ed9 [CANDIDATE]: Term 1 election: Requested vote from peers 7ad5c7bfe7a14c3fb4408d6ed8a11057 (127.3.61.190:41645), d72d675a81024955a8aa1324aae4782f (127.3.61.188:43463)
I20260811 19:14:56.444115 26620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "581dcb09f4374097938f97b6013c6ed9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72d675a81024955a8aa1324aae4782f"
I20260811 19:14:56.444378 26620 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d72d675a81024955a8aa1324aae4782f [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:14:56.448400 26620 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d72d675a81024955a8aa1324aae4782f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 581dcb09f4374097938f97b6013c6ed9 in term 1.
I20260811 19:14:56.449133 26469 leader_election.cc:302] T 00000000000000000000000000000000 P 581dcb09f4374097938f97b6013c6ed9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 581dcb09f4374097938f97b6013c6ed9, d72d675a81024955a8aa1324aae4782f; no voters:
I20260811 19:14:56.449705 26702 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 581dcb09f4374097938f97b6013c6ed9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:14:56.455435 26721 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ad5c7bfe7a14c3fb4408d6ed8a11057 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ad5c7bfe7a14c3fb4408d6ed8a11057" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 41645 } } peers { permanent_uuid: "581dcb09f4374097938f97b6013c6ed9" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43763 } } peers { permanent_uuid: "d72d675a81024955a8aa1324aae4782f" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 43463 } } }
I20260811 19:14:56.457998 26429 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ad5c7bfe7a14c3fb4408d6ed8a11057 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:14:56.458066 26397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "581dcb09f4374097938f97b6013c6ed9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ad5c7bfe7a14c3fb4408d6ed8a11057" is_pre_election: true
I20260811 19:14:56.458834 26397 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7ad5c7bfe7a14c3fb4408d6ed8a11057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 581dcb09f4374097938f97b6013c6ed9 in term 0.
I20260811 19:14:56.459306 26396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "581dcb09f4374097938f97b6013c6ed9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ad5c7bfe7a14c3fb4408d6ed8a11057"
I20260811 19:14:56.459599 26396 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ad5c7bfe7a14c3fb4408d6ed8a11057 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:14:56.458164 26721 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ad5c7bfe7a14c3fb4408d6ed8a11057 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:14:56.463295 26396 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7ad5c7bfe7a14c3fb4408d6ed8a11057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 581dcb09f4374097938f97b6013c6ed9 in term 1.
I20260811 19:14:56.468135 26702 raft_consensus.cc:686] T 00000000000000000000000000000000 P 581dcb09f4374097938f97b6013c6ed9 [term 1 LEADER]: Becoming Leader. State: Replica: 581dcb09f4374097938f97b6013c6ed9, State: Running, Role: LEADER
I20260811 19:14:56.468884 26702 consensus_queue.cc:237] T 00000000000000000000000000000000 P 581dcb09f4374097938f97b6013c6ed9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ad5c7bfe7a14c3fb4408d6ed8a11057" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 41645 } } peers { permanent_uuid: "581dcb09f4374097938f97b6013c6ed9" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43763 } } peers { permanent_uuid: "d72d675a81024955a8aa1324aae4782f" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 43463 } }
I20260811 19:14:56.480278 26722 sys_catalog.cc:455] T 00000000000000000000000000000000 P 581dcb09f4374097938f97b6013c6ed9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 581dcb09f4374097938f97b6013c6ed9. Latest consensus state: current_term: 1 leader_uuid: "581dcb09f4374097938f97b6013c6ed9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ad5c7bfe7a14c3fb4408d6ed8a11057" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 41645 } } peers { permanent_uuid: "581dcb09f4374097938f97b6013c6ed9" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43763 } } peers { permanent_uuid: "d72d675a81024955a8aa1324aae4782f" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 43463 } } }
I20260811 19:14:56.480774 26722 sys_catalog.cc:458] T 00000000000000000000000000000000 P 581dcb09f4374097938f97b6013c6ed9 [sys.catalog]: This master's current role is: LEADER
I20260811 19:14:56.493849 26732 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:14:56.497251 26732 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:14:56.519251 26620 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d72d675a81024955a8aa1324aae4782f [term 1 FOLLOWER]: Refusing update from remote peer 581dcb09f4374097938f97b6013c6ed9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:14:56.520519 26722 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 581dcb09f4374097938f97b6013c6ed9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72d675a81024955a8aa1324aae4782f" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 43463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:14:56.525218 26741 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 19:14:56.527539 26396 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7ad5c7bfe7a14c3fb4408d6ed8a11057 [term 1 FOLLOWER]: Refusing update from remote peer 581dcb09f4374097938f97b6013c6ed9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:14:56.528465 26722 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 581dcb09f4374097938f97b6013c6ed9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ad5c7bfe7a14c3fb4408d6ed8a11057" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 41645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 19:14:56.539881 26472 tablet.cc:2369] T 00000000000000000000000000000000 P 581dcb09f4374097938f97b6013c6ed9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:14:56.542217 26721 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ad5c7bfe7a14c3fb4408d6ed8a11057 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 581dcb09f4374097938f97b6013c6ed9. Latest consensus state: current_term: 1 leader_uuid: "581dcb09f4374097938f97b6013c6ed9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ad5c7bfe7a14c3fb4408d6ed8a11057" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 41645 } } peers { permanent_uuid: "581dcb09f4374097938f97b6013c6ed9" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43763 } } peers { permanent_uuid: "d72d675a81024955a8aa1324aae4782f" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 43463 } } }
I20260811 19:14:56.542611 26721 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ad5c7bfe7a14c3fb4408d6ed8a11057 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:14:56.547168 26738 mvcc.cc:205] Tried to move back new op lower bound from 7317404452930449408 to 7317404452742258688. Current Snapshot: MvccSnapshot[applied={T|T < 7317404452930449408}]
I20260811 19:14:56.548887 26682 sys_catalog.cc:455] T 00000000000000000000000000000000 P d72d675a81024955a8aa1324aae4782f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 581dcb09f4374097938f97b6013c6ed9. Latest consensus state: current_term: 1 leader_uuid: "581dcb09f4374097938f97b6013c6ed9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ad5c7bfe7a14c3fb4408d6ed8a11057" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 41645 } } peers { permanent_uuid: "581dcb09f4374097938f97b6013c6ed9" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43763 } } peers { permanent_uuid: "d72d675a81024955a8aa1324aae4782f" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 43463 } } }
I20260811 19:14:56.549258 26682 sys_catalog.cc:458] T 00000000000000000000000000000000 P d72d675a81024955a8aa1324aae4782f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:14:56.552855 26722 sys_catalog.cc:455] T 00000000000000000000000000000000 P 581dcb09f4374097938f97b6013c6ed9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "581dcb09f4374097938f97b6013c6ed9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ad5c7bfe7a14c3fb4408d6ed8a11057" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 41645 } } peers { permanent_uuid: "581dcb09f4374097938f97b6013c6ed9" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43763 } } peers { permanent_uuid: "d72d675a81024955a8aa1324aae4782f" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 43463 } } }
I20260811 19:14:56.553246 26722 sys_catalog.cc:458] T 00000000000000000000000000000000 P 581dcb09f4374097938f97b6013c6ed9 [sys.catalog]: This master's current role is: LEADER
I20260811 19:14:56.554674 26702 sys_catalog.cc:455] T 00000000000000000000000000000000 P 581dcb09f4374097938f97b6013c6ed9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "581dcb09f4374097938f97b6013c6ed9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ad5c7bfe7a14c3fb4408d6ed8a11057" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 41645 } } peers { permanent_uuid: "581dcb09f4374097938f97b6013c6ed9" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43763 } } peers { permanent_uuid: "d72d675a81024955a8aa1324aae4782f" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 43463 } } }
I20260811 19:14:56.555045 26702 sys_catalog.cc:458] T 00000000000000000000000000000000 P 581dcb09f4374097938f97b6013c6ed9 [sys.catalog]: This master's current role is: LEADER
I20260811 19:14:56.565981 26732 catalog_manager.cc:1347] Generated new cluster ID: bdb09476d1284c808f845e8c9f7bad97
I20260811 19:14:56.566135 26732 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:14:56.574640 26682 sys_catalog.cc:455] T 00000000000000000000000000000000 P d72d675a81024955a8aa1324aae4782f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "581dcb09f4374097938f97b6013c6ed9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ad5c7bfe7a14c3fb4408d6ed8a11057" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 41645 } } peers { permanent_uuid: "581dcb09f4374097938f97b6013c6ed9" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43763 } } peers { permanent_uuid: "d72d675a81024955a8aa1324aae4782f" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 43463 } } }
I20260811 19:14:56.574988 26682 sys_catalog.cc:458] T 00000000000000000000000000000000 P d72d675a81024955a8aa1324aae4782f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:14:56.580556 26721 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ad5c7bfe7a14c3fb4408d6ed8a11057 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "581dcb09f4374097938f97b6013c6ed9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ad5c7bfe7a14c3fb4408d6ed8a11057" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 41645 } } peers { permanent_uuid: "581dcb09f4374097938f97b6013c6ed9" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 43763 } } peers { permanent_uuid: "d72d675a81024955a8aa1324aae4782f" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 43463 } } }
I20260811 19:14:56.580883 26721 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ad5c7bfe7a14c3fb4408d6ed8a11057 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:14:56.591480 26732 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:14:56.595350 26732 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:14:56.616449 26732 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 581dcb09f4374097938f97b6013c6ed9: Generated new TSK 0
I20260811 19:14:56.619671 26732 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:14:57.294546 26697 catalog_manager.cc:1259] Loaded cluster ID: bdb09476d1284c808f845e8c9f7bad97
I20260811 19:14:57.294693 26697 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d72d675a81024955a8aa1324aae4782f: loading cluster ID for follower catalog manager: success
I20260811 19:14:57.297482 26697 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d72d675a81024955a8aa1324aae4782f: acquiring CA information for follower catalog manager: success
I20260811 19:14:57.299489 26697 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d72d675a81024955a8aa1324aae4782f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 19:14:57.531447 26741 catalog_manager.cc:1259] Loaded cluster ID: bdb09476d1284c808f845e8c9f7bad97
I20260811 19:14:57.531646 26741 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7ad5c7bfe7a14c3fb4408d6ed8a11057: loading cluster ID for follower catalog manager: success
I20260811 19:14:57.535180 26741 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7ad5c7bfe7a14c3fb4408d6ed8a11057: acquiring CA information for follower catalog manager: success
I20260811 19:14:57.537998 26741 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7ad5c7bfe7a14c3fb4408d6ed8a11057: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 19:14:57.713992 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 26324
I20260811 19:14:57.715497 26424 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:14:57.837841 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 26324
I20260811 19:14:57.874258 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 26439
I20260811 19:14:57.875702 26543 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:14:57.996289 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 26439
I20260811 19:14:58.028842 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 26560
I20260811 19:14:58.030172 26650 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:14:58.146083 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 26560
2026-08-11T19:14:58Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (5427 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786475454804547-3318-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:14:58 dist-test-slave-t7g5 krb5kdc[26763](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:14:58 dist-test-slave-t7g5 krb5kdc[26763](info): set up 2 sockets
Aug 11 19:14:58 dist-test-slave-t7g5 krb5kdc[26763](info): commencing operation
krb5kdc: starting...
W20260811 19:14:59.510115 3318 mini_kdc.cc:121] Time spent starting KDC: real 1.309s user 0.000s sys 0.056s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Aug 11 19:14:59 dist-test-slave-t7g5 krb5kdc[26763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:14:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:14:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.61.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab.
Entry for principal kudu/127.3.61.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab.
I20260811 19:14:59.866736 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786475454804547-3318-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786475454804547-3318-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786475454804547-3318-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:34033
--webserver_interface=127.3.61.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:44307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.61.190:34033
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:15:00.262516 26780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:15:00.262859 26780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:15:00.262981 26780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:15:00.270992 26780 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:15:00.271144 26780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:15:00.271209 26780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:15:00.271293 26780 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:15:00.271370 26780 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:15:00.282225 26780 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:44307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786475454804547-3318-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786475454804547-3318-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.61.190:34033
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:34033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.61.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786475454804547-3318-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:15:00.283720 26780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:15:00.285100 26780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:15:00.298671 26787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:15:00.298856 26788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:15:00.299474 26791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:15:00.300283 26780 server_base.cc:1034] running on GCE node
I20260811 19:15:00.301268 26780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:15:00.302897 26780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:15:00.304170 26780 hybrid_clock.cc:648] HybridClock initialized: now 1786475700304099 us; error 45 us; skew 500 ppm
I20260811 19:15:00.304587 26780 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:15:00 dist-test-slave-t7g5 krb5kdc[26763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475700, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:15:00.308089 26780 init.cc:377] Logged in from keytab as kudu/127.3.61.190@KRBTEST.COM (short username kudu)
I20260811 19:15:00.309851 26780 webserver.cc:460] Webserver started at http://127.3.61.190:46673/ using document root <none> and password file <none>
I20260811 19:15:00.310602 26780 fs_manager.cc:359] Metadata directory not provided
I20260811 19:15:00.310729 26780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:15:00.311062 26780 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:15:00.314481 26780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786475454804547-3318-0/minicluster-data/master-0/data/instance:
uuid: "cc9a218f9bfd4e808963d4a6e379100d"
format_stamp: "Formatted at 2026-08-11 19:15:00 on dist-test-slave-t7g5"
server_key: "712dc33b160ed0899131b0437e436c7d"
server_key_iv: "81ce686cc9cbb6c24a1ca0a1f1b97ad2"
server_key_version: "encryptionkey@0"
I20260811 19:15:00.315871 26780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786475454804547-3318-0/minicluster-data/master-0/wal/instance:
uuid: "cc9a218f9bfd4e808963d4a6e379100d"
format_stamp: "Formatted at 2026-08-11 19:15:00 on dist-test-slave-t7g5"
server_key: "712dc33b160ed0899131b0437e436c7d"
server_key_iv: "81ce686cc9cbb6c24a1ca0a1f1b97ad2"
server_key_version: "encryptionkey@0"
I20260811 19:15:00.321928 26780 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260811 19:15:00.325912 26802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:15:00.327458 26780 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 19:15:00.327656 26780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786475454804547-3318-0/minicluster-data/master-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786475454804547-3318-0/minicluster-data/master-0/wal
uuid: "cc9a218f9bfd4e808963d4a6e379100d"
format_stamp: "Formatted at 2026-08-11 19:15:00 on dist-test-slave-t7g5"
server_key: "712dc33b160ed0899131b0437e436c7d"
server_key_iv: "81ce686cc9cbb6c24a1ca0a1f1b97ad2"
server_key_version: "encryptionkey@0"
I20260811 19:15:00.327868 26780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786475454804547-3318-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786475454804547-3318-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786475454804547-3318-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:15:00.347755 26780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:15:00.353947 26780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:15:00.354347 26780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:15:00.377465 26780 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.190:34033
I20260811 19:15:00.377493 26892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.190:34033 every 8 connection(s)
I20260811 19:15:00.379416 26780 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
I20260811 19:15:00.383297 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 26780
I20260811 19:15:00.383581 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786475454804547-3318-0/minicluster-data/master-0/wal/instance
I20260811 19:15:00.384439 26893 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:15:00.384465 3318 external_mini_cluster.cc:1428] Setting key 5b07e9113c24faa3bb1b9a6954694657
I20260811 19:15:00.401432 26893 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc9a218f9bfd4e808963d4a6e379100d: Bootstrap starting.
Aug 11 19:15:00 dist-test-slave-t7g5 krb5kdc[26763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:15:00.405685 26893 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cc9a218f9bfd4e808963d4a6e379100d: Neither blocks nor log segments found. Creating new log.
I20260811 19:15:00.406913 26893 log.cc:824] T 00000000000000000000000000000000 P cc9a218f9bfd4e808963d4a6e379100d: Log is configured to *not* fsync() on all Append() calls
I20260811 19:15:00.411455 26893 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc9a218f9bfd4e808963d4a6e379100d: No bootstrap required, opened a new log
I20260811 19:15:00.414791 26896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:00.386559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41800 (local address 127.3.61.190:34033)
0811 19:15:00.387402 (+ 843us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:00.387416 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:00.387463 (+ 47us) server_negotiation.cc:407] Connection header received
0811 19:15:00.390422 (+ 2959us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:00.390454 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:00.390986 (+ 532us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:00.391398 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:15:00.393627 (+ 2229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:00.395534 (+ 1907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:00.397740 (+ 2206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:00.398692 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:00.398872 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:00.404952 (+ 6080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:15:00.404964 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:15:00.404983 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:15:00.405047 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:15:00.408797 (+ 3750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:00.410363 (+ 1566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:00.410371 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:00.410387 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:00.410610 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:00.411336 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:00.411341 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:00.411343 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:00.412069 (+ 726us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:15:00.412341 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:00.413069 (+ 728us) server_negotiation.cc:299] Negotiation successful
0811 19:15:00.413529 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":617,"thread_start_us":374,"threads_started":1}
I20260811 19:15:00.419697 3318 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:15:00.422995 26893 raft_consensus.cc:348] T 00000000000000000000000000000000 P cc9a218f9bfd4e808963d4a6e379100d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc9a218f9bfd4e808963d4a6e379100d" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 34033 } }
I20260811 19:15:00.423437 26893 raft_consensus.cc:374] T 00000000000000000000000000000000 P cc9a218f9bfd4e808963d4a6e379100d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:15:00.423549 26893 raft_consensus.cc:729] T 00000000000000000000000000000000 P cc9a218f9bfd4e808963d4a6e379100d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc9a218f9bfd4e808963d4a6e379100d, State: Initialized, Role: FOLLOWER
I20260811 19:15:00.424146 26893 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc9a218f9bfd4e808963d4a6e379100d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc9a218f9bfd4e808963d4a6e379100d" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 34033 } }
I20260811 19:15:00.424392 26893 raft_consensus.cc:388] T 00000000000000000000000000000000 P cc9a218f9bfd4e808963d4a6e379100d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:15:00.424538 26893 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc9a218f9bfd4e808963d4a6e379100d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:15:00.424711 26893 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc9a218f9bfd4e808963d4a6e379100d [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:15:00.427055 26893 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc9a218f9bfd4e808963d4a6e379100d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc9a218f9bfd4e808963d4a6e379100d" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 34033 } }
I20260811 19:15:00.427595 26893 leader_election.cc:302] T 00000000000000000000000000000000 P cc9a218f9bfd4e808963d4a6e379100d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cc9a218f9bfd4e808963d4a6e379100d; no voters:
I20260811 19:15:00.428208 26893 leader_election.cc:288] T 00000000000000000000000000000000 P cc9a218f9bfd4e808963d4a6e379100d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:15:00.428350 26900 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cc9a218f9bfd4e808963d4a6e379100d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:15:00.428996 26900 raft_consensus.cc:686] T 00000000000000000000000000000000 P cc9a218f9bfd4e808963d4a6e379100d [term 1 LEADER]: Becoming Leader. State: Replica: cc9a218f9bfd4e808963d4a6e379100d, State: Running, Role: LEADER
I20260811 19:15:00.429554 26900 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc9a218f9bfd4e808963d4a6e379100d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc9a218f9bfd4e808963d4a6e379100d" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 34033 } }
I20260811 19:15:00.430400 26893 sys_catalog.cc:564] T 00000000000000000000000000000000 P cc9a218f9bfd4e808963d4a6e379100d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:15:00.432485 26903 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc9a218f9bfd4e808963d4a6e379100d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cc9a218f9bfd4e808963d4a6e379100d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc9a218f9bfd4e808963d4a6e379100d" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 34033 } } }
I20260811 19:15:00.432514 26904 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc9a218f9bfd4e808963d4a6e379100d [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc9a218f9bfd4e808963d4a6e379100d. Latest consensus state: current_term: 1 leader_uuid: "cc9a218f9bfd4e808963d4a6e379100d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc9a218f9bfd4e808963d4a6e379100d" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 34033 } } }
I20260811 19:15:00.432967 26903 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc9a218f9bfd4e808963d4a6e379100d [sys.catalog]: This master's current role is: LEADER
I20260811 19:15:00.432967 26904 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc9a218f9bfd4e808963d4a6e379100d [sys.catalog]: This master's current role is: LEADER
I20260811 19:15:00.433982 26909 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:15:00.442234 26909 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:15:00.452172 26909 catalog_manager.cc:1347] Generated new cluster ID: 056d53f3003b40159f046c7772ef7d19
I20260811 19:15:00.452318 26909 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:15:00.472415 26909 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:15:00.474576 26909 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:15:00.486606 26909 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cc9a218f9bfd4e808963d4a6e379100d: Generated new TSK 0
I20260811 19:15:00.487905 26909 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:15:00.535732 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 26780
I20260811 19:15:00.537367 26888 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:15:00.639961 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 26780
2026-08-11T19:15:00Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2490 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:15:00 dist-test-slave-t7g5 krb5kdc[26939](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:15:00 dist-test-slave-t7g5 krb5kdc[26939](info): set up 2 sockets
Aug 11 19:15:00 dist-test-slave-t7g5 krb5kdc[26939](info): commencing operation
krb5kdc: starting...
W20260811 19:15:01.976781 3318 mini_kdc.cc:121] Time spent starting KDC: real 1.286s user 0.007s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Aug 11 19:15:02 dist-test-slave-t7g5 krb5kdc[26939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475702, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:15:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:15:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.61.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab.
Entry for principal kudu/127.3.61.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab.
I20260811 19:15:02.326021 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:35949
--webserver_interface=127.3.61.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:42151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.61.190:35949
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:15:02.710135 26959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:15:02.710475 26959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:15:02.710613 26959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:15:02.718291 26959 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:15:02.718430 26959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:15:02.718513 26959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:15:02.718605 26959 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:15:02.718647 26959 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:15:02.729719 26959 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:42151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.61.190:35949
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:35949
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.61.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:15:02.732059 26959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:15:02.733434 26959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:15:02.746187 26968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:15:02.746567 26969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:15:02.746584 26972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:15:02.747730 26959 server_base.cc:1034] running on GCE node
I20260811 19:15:02.748620 26959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:15:02.750294 26959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:15:02.751547 26959 hybrid_clock.cc:648] HybridClock initialized: now 1786475702751481 us; error 48 us; skew 500 ppm
I20260811 19:15:02.751920 26959 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:15:02 dist-test-slave-t7g5 krb5kdc[26939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475702, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:15:02.755409 26959 init.cc:377] Logged in from keytab as kudu/127.3.61.190@KRBTEST.COM (short username kudu)
I20260811 19:15:02.757289 26959 webserver.cc:460] Webserver started at http://127.3.61.190:33277/ using document root <none> and password file <none>
I20260811 19:15:02.758077 26959 fs_manager.cc:359] Metadata directory not provided
I20260811 19:15:02.758198 26959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:15:02.758493 26959 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:15:02.761617 26959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/master-0/data/instance:
uuid: "053ea5d60afa4ce4a2e5e9d1d6ee97fd"
format_stamp: "Formatted at 2026-08-11 19:15:02 on dist-test-slave-t7g5"
server_key: "9486a0c26b1a1ebd6229783aad0bb6db"
server_key_iv: "f0c51706b27670ff1cf0a4a7ee1778a5"
server_key_version: "encryptionkey@0"
I20260811 19:15:02.762441 26959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/master-0/wal/instance:
uuid: "053ea5d60afa4ce4a2e5e9d1d6ee97fd"
format_stamp: "Formatted at 2026-08-11 19:15:02 on dist-test-slave-t7g5"
server_key: "9486a0c26b1a1ebd6229783aad0bb6db"
server_key_iv: "f0c51706b27670ff1cf0a4a7ee1778a5"
server_key_version: "encryptionkey@0"
I20260811 19:15:02.768968 26959 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260811 19:15:02.773258 26981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:15:02.774864 26959 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:15:02.775135 26959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/master-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/master-0/wal
uuid: "053ea5d60afa4ce4a2e5e9d1d6ee97fd"
format_stamp: "Formatted at 2026-08-11 19:15:02 on dist-test-slave-t7g5"
server_key: "9486a0c26b1a1ebd6229783aad0bb6db"
server_key_iv: "f0c51706b27670ff1cf0a4a7ee1778a5"
server_key_version: "encryptionkey@0"
I20260811 19:15:02.775377 26959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:15:02.795171 26959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:15:02.799578 26959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:15:02.799862 26959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:15:02.821388 26959 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.190:35949
I20260811 19:15:02.821470 27048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.190:35949 every 8 connection(s)
I20260811 19:15:02.822824 26959 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
I20260811 19:15:02.826684 27049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:15:02.831861 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 26959
I20260811 19:15:02.832154 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/master-0/wal/instance
I20260811 19:15:02.832909 3318 external_mini_cluster.cc:1428] Setting key beac8ae8413034974803521087219cf1
I20260811 19:15:02.841531 27049 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd: Bootstrap starting.
I20260811 19:15:02.845296 27049 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd: Neither blocks nor log segments found. Creating new log.
I20260811 19:15:02.846678 27049 log.cc:824] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd: Log is configured to *not* fsync() on all Append() calls
I20260811 19:15:02.849792 27049 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd: No bootstrap required, opened a new log
Aug 11 19:15:02 dist-test-slave-t7g5 krb5kdc[26939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475702, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:15:02.860771 27049 raft_consensus.cc:348] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "053ea5d60afa4ce4a2e5e9d1d6ee97fd" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35949 } }
I20260811 19:15:02.861239 27049 raft_consensus.cc:374] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:15:02.861344 27049 raft_consensus.cc:729] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 053ea5d60afa4ce4a2e5e9d1d6ee97fd, State: Initialized, Role: FOLLOWER
I20260811 19:15:02.862053 27057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:02.835026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41968 (local address 127.3.61.190:35949)
0811 19:15:02.836051 (+ 1025us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:02.836072 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:02.836146 (+ 74us) server_negotiation.cc:407] Connection header received
0811 19:15:02.839604 (+ 3458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:02.839641 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:02.840179 (+ 538us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:02.840656 (+ 477us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:15:02.842003 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:02.843980 (+ 1977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:02.845515 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:02.846591 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:02.846787 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:02.851628 (+ 4841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:15:02.851669 (+ 41us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:15:02.851707 (+ 38us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:15:02.851780 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:15:02.855747 (+ 3967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:02.857202 (+ 1455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:02.857210 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:02.857218 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:02.857537 (+ 319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:02.858161 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:02.858167 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:02.858169 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:02.859135 (+ 966us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:15:02.859477 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:02.860248 (+ 771us) server_negotiation.cc:299] Negotiation successful
0811 19:15:02.860724 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":691,"thread_start_us":400,"threads_started":1}
I20260811 19:15:02.862015 27049 consensus_queue.cc:260] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "053ea5d60afa4ce4a2e5e9d1d6ee97fd" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35949 } }
I20260811 19:15:02.862300 27049 raft_consensus.cc:388] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:15:02.862463 27049 raft_consensus.cc:482] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:15:02.862670 27049 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:15:02.865581 27049 raft_consensus.cc:504] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "053ea5d60afa4ce4a2e5e9d1d6ee97fd" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35949 } }
I20260811 19:15:02.866190 27049 leader_election.cc:302] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 053ea5d60afa4ce4a2e5e9d1d6ee97fd; no voters:
I20260811 19:15:02.866819 27049 leader_election.cc:288] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:15:02.866942 27061 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:15:02.867673 27061 raft_consensus.cc:686] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd [term 1 LEADER]: Becoming Leader. State: Replica: 053ea5d60afa4ce4a2e5e9d1d6ee97fd, State: Running, Role: LEADER
I20260811 19:15:02.868257 27061 consensus_queue.cc:237] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "053ea5d60afa4ce4a2e5e9d1d6ee97fd" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35949 } }
I20260811 19:15:02.869035 27049 sys_catalog.cc:564] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:15:02.871012 27065 sys_catalog.cc:455] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 053ea5d60afa4ce4a2e5e9d1d6ee97fd. Latest consensus state: current_term: 1 leader_uuid: "053ea5d60afa4ce4a2e5e9d1d6ee97fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "053ea5d60afa4ce4a2e5e9d1d6ee97fd" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35949 } } }
I20260811 19:15:02.871388 27065 sys_catalog.cc:458] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd [sys.catalog]: This master's current role is: LEADER
I20260811 19:15:02.871959 27064 sys_catalog.cc:455] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "053ea5d60afa4ce4a2e5e9d1d6ee97fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "053ea5d60afa4ce4a2e5e9d1d6ee97fd" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35949 } } }
I20260811 19:15:02.872334 27064 sys_catalog.cc:458] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd [sys.catalog]: This master's current role is: LEADER
I20260811 19:15:02.876245 27075 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:15:02.883168 27075 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:15:02.893446 27075 catalog_manager.cc:1347] Generated new cluster ID: c718e517bf764f458269c35efd63348a
I20260811 19:15:02.893590 27075 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:15:02.915506 27075 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:15:02.918011 27075 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:15:02.932072 27075 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd: Generated new TSK 0
I20260811 19:15:02.933043 27075 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.61.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab.
Entry for principal kudu/127.3.61.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab.
I20260811 19:15:03.024089 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.129:0
--local_ip_for_outbound_sockets=127.3.61.129
--webserver_interface=127.3.61.129
--webserver_port=0
--tserver_master_addrs=127.3.61.190:35949
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab
--principal=kudu/127.3.61.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:42151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:15:03.426692 27097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:15:03.427045 27097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:15:03.427147 27097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:15:03.436075 27097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:15:03.436215 27097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:15:03.436335 27097 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.129
I20260811 19:15:03.447948 27097 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:42151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab
--principal=kudu/127.3.61.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.61.129
--webserver_port=0
--tserver_master_addrs=127.3.61.190:35949
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.129
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:15:03.449568 27097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:15:03.451114 27097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:15:03.466234 27108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:15:03.466217 27104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:15:03.466348 27105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:15:03.467156 27097 server_base.cc:1034] running on GCE node
I20260811 19:15:03.467859 27097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:15:03.468935 27097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:15:03.470193 27097 hybrid_clock.cc:648] HybridClock initialized: now 1786475703470124 us; error 65 us; skew 500 ppm
I20260811 19:15:03.470561 27097 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:15:03 dist-test-slave-t7g5 krb5kdc[26939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475703, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:15:03.478348 27097 init.cc:377] Logged in from keytab as kudu/127.3.61.129@KRBTEST.COM (short username kudu)
I20260811 19:15:03.480870 27097 webserver.cc:460] Webserver started at http://127.3.61.129:45241/ using document root <none> and password file <none>
I20260811 19:15:03.481714 27097 fs_manager.cc:359] Metadata directory not provided
I20260811 19:15:03.481863 27097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:15:03.482201 27097 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:15:03.485602 27097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-0/data/instance:
uuid: "c5948773534942afa0c14b4ef53aff64"
format_stamp: "Formatted at 2026-08-11 19:15:03 on dist-test-slave-t7g5"
server_key: "aa2e104d864826a3228fed8e38f88af1"
server_key_iv: "037da0fbd9387d24fcc99627c8a0165a"
server_key_version: "encryptionkey@0"
I20260811 19:15:03.486805 27097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-0/wal/instance:
uuid: "c5948773534942afa0c14b4ef53aff64"
format_stamp: "Formatted at 2026-08-11 19:15:03 on dist-test-slave-t7g5"
server_key: "aa2e104d864826a3228fed8e38f88af1"
server_key_iv: "037da0fbd9387d24fcc99627c8a0165a"
server_key_version: "encryptionkey@0"
I20260811 19:15:03.492859 27097 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 19:15:03.497300 27118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:15:03.499424 27097 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 19:15:03.499647 27097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-0/wal
uuid: "c5948773534942afa0c14b4ef53aff64"
format_stamp: "Formatted at 2026-08-11 19:15:03 on dist-test-slave-t7g5"
server_key: "aa2e104d864826a3228fed8e38f88af1"
server_key_iv: "037da0fbd9387d24fcc99627c8a0165a"
server_key_version: "encryptionkey@0"
I20260811 19:15:03.499895 27097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:15:03.521582 27097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:15:03.526469 27097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:15:03.526796 27097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:15:03.527935 27097 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:15:03.529524 27097 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:15:03.529613 27097 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:15:03.529784 27097 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:15:03.529839 27097 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:15:03.565404 27097 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.129:43955
I20260811 19:15:03.565459 27279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.129:43955 every 8 connection(s)
I20260811 19:15:03.566793 27097 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
I20260811 19:15:03.576351 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 27097
I20260811 19:15:03.576635 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-0/wal/instance
I20260811 19:15:03.577392 3318 external_mini_cluster.cc:1428] Setting key 80043a67ac620c8908a5c7a412d2a0db
Aug 11 19:15:03 dist-test-slave-t7g5 krb5kdc[26939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475703, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.129@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:15:03.589908 27283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:03.569817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:50981 (local address 127.3.61.190:35949)
0811 19:15:03.570298 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:03.570304 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:03.571262 (+ 958us) server_negotiation.cc:407] Connection header received
0811 19:15:03.573192 (+ 1930us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:03.573197 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:03.573283 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:03.573383 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:15:03.575083 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:03.575851 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:03.577530 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:03.577971 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:03.578088 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:03.583333 (+ 5245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:15:03.583339 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:15:03.583342 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:15:03.583379 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:15:03.586459 (+ 3080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:03.587328 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:03.587333 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:03.587335 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:03.587475 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:03.587935 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:03.587938 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:03.587940 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:03.588226 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:15:03.588369 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:03.589206 (+ 837us) server_negotiation.cc:299] Negotiation successful
0811 19:15:03.589483 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":237,"threads_started":1}
I20260811 19:15:03.591143 27286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:03.570275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35949 (local address 127.3.61.129:50981)
0811 19:15:03.571083 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:03.571123 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 19:15:03.572876 (+ 1753us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:15:03.573719 (+ 843us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:15:03.573732 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:15:03.574297 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:15:03.574896 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:03.574926 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:03.576018 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:03.576022 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:03.577371 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:03.577386 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:03.578217 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:03.578224 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:03.578508 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:03.579421 (+ 913us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:15:03.579465 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:15:03.583045 (+ 3580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:15:03.586646 (+ 3601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:15:03.586658 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:15:03.586675 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:15:03.587178 (+ 503us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:15:03.587590 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:15:03.587595 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:15:03.587598 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:15:03.587806 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:15:03.588365 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:15:03.588375 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:15:03.588770 (+ 395us) client_negotiation.cc:769] Sending connection context
0811 19:15:03.589071 (+ 301us) client_negotiation.cc:240] Negotiation successful
0811 19:15:03.589701 (+ 630us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":323,"threads_started":1}
I20260811 19:15:03.592942 27280 heartbeater.cc:347] Connected to a master server at 127.3.61.190:35949
I20260811 19:15:03.593253 27280 heartbeater.cc:464] Registering TS with master...
I20260811 19:15:03.594080 27280 heartbeater.cc:511] Master 127.3.61.190:35949 requested a full tablet report, sending...
I20260811 19:15:03.596462 27005 ts_manager.cc:194] Registered new tserver with Master: c5948773534942afa0c14b4ef53aff64 (127.3.61.129:43955)
I20260811 19:15:03.598080 27005 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.129@KRBTEST.COM'} at 127.3.61.129:50981
WARNING: no policy specified for kudu/127.3.61.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab.
Entry for principal kudu/127.3.61.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab.
I20260811 19:15:03.650281 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.130:0
--local_ip_for_outbound_sockets=127.3.61.130
--webserver_interface=127.3.61.130
--webserver_port=0
--tserver_master_addrs=127.3.61.190:35949
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab
--principal=kudu/127.3.61.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:42151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:15:04.044709 27289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:15:04.045060 27289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:15:04.045238 27289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:15:04.056924 27289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:15:04.057090 27289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:15:04.057264 27289 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.130
I20260811 19:15:04.070817 27289 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:42151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab
--principal=kudu/127.3.61.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.61.130
--webserver_port=0
--tserver_master_addrs=127.3.61.190:35949
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.130
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:15:04.072317 27289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:15:04.073815 27289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:15:04.088013 27295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:15:04.088028 27296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:15:04.088002 27299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:15:04.089373 27289 server_base.cc:1034] running on GCE node
I20260811 19:15:04.090114 27289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:15:04.091435 27289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:15:04.092823 27289 hybrid_clock.cc:648] HybridClock initialized: now 1786475704092750 us; error 48 us; skew 500 ppm
I20260811 19:15:04.093189 27289 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:15:04 dist-test-slave-t7g5 krb5kdc[26939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475704, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:15:04.096794 27289 init.cc:377] Logged in from keytab as kudu/127.3.61.130@KRBTEST.COM (short username kudu)
I20260811 19:15:04.098441 27289 webserver.cc:460] Webserver started at http://127.3.61.130:35339/ using document root <none> and password file <none>
I20260811 19:15:04.099189 27289 fs_manager.cc:359] Metadata directory not provided
I20260811 19:15:04.099326 27289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:15:04.099648 27289 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:15:04.102869 27289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-1/data/instance:
uuid: "40aa419517fc4e9db6457742b1257881"
format_stamp: "Formatted at 2026-08-11 19:15:04 on dist-test-slave-t7g5"
server_key: "184bba89543eb0ebf583f36b6d44c07d"
server_key_iv: "593a93631308d45ab515262433bcf256"
server_key_version: "encryptionkey@0"
I20260811 19:15:04.103690 27289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-1/wal/instance:
uuid: "40aa419517fc4e9db6457742b1257881"
format_stamp: "Formatted at 2026-08-11 19:15:04 on dist-test-slave-t7g5"
server_key: "184bba89543eb0ebf583f36b6d44c07d"
server_key_iv: "593a93631308d45ab515262433bcf256"
server_key_version: "encryptionkey@0"
I20260811 19:15:04.109742 27289 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260811 19:15:04.114430 27309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:15:04.116127 27289 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260811 19:15:04.116331 27289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-1/wal
uuid: "40aa419517fc4e9db6457742b1257881"
format_stamp: "Formatted at 2026-08-11 19:15:04 on dist-test-slave-t7g5"
server_key: "184bba89543eb0ebf583f36b6d44c07d"
server_key_iv: "593a93631308d45ab515262433bcf256"
server_key_version: "encryptionkey@0"
I20260811 19:15:04.116572 27289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:15:04.132861 27289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:15:04.137919 27289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:15:04.138310 27289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:15:04.139345 27289 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:15:04.140815 27289 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:15:04.140898 27289 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:15:04.141021 27289 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:15:04.141079 27289 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:15:04.178870 27289 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.130:38049
I20260811 19:15:04.178887 27474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.130:38049 every 8 connection(s)
I20260811 19:15:04.180702 27289 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
I20260811 19:15:04.186861 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 27289
I20260811 19:15:04.187076 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-1/wal/instance
I20260811 19:15:04.187582 3318 external_mini_cluster.cc:1428] Setting key 326190a37e149ac1dfa9d941476eea57
Aug 11 19:15:04 dist-test-slave-t7g5 krb5kdc[26939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475704, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.130@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:15:04.209079 27480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:04.183597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:40773 (local address 127.3.61.190:35949)
0811 19:15:04.184082 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:04.184087 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:04.185578 (+ 1491us) server_negotiation.cc:407] Connection header received
0811 19:15:04.187626 (+ 2048us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:04.187632 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:04.187722 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:04.187821 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:15:04.190384 (+ 2563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:04.191974 (+ 1590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:04.194618 (+ 2644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:04.195253 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:04.195381 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:04.201069 (+ 5688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:15:04.201086 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:15:04.201092 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:15:04.201152 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:15:04.204722 (+ 3570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:04.206024 (+ 1302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:04.206030 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:04.206034 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:04.206191 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:04.206804 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:04.206811 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:04.206814 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:04.207377 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:15:04.207524 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:04.208541 (+ 1017us) server_negotiation.cc:299] Negotiation successful
0811 19:15:04.208754 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":252,"threads_started":1}
I20260811 19:15:04.210510 27481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:04.184414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35949 (local address 127.3.61.130:40773)
0811 19:15:04.185392 (+ 978us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:04.185442 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 19:15:04.187332 (+ 1890us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:15:04.188106 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:15:04.188120 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:15:04.188943 (+ 823us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:15:04.189892 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:04.189924 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:04.192283 (+ 2359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:04.192290 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:04.194372 (+ 2082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:04.194393 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:04.195626 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:04.195635 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:04.196012 (+ 377us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:04.197178 (+ 1166us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:15:04.197214 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:15:04.200409 (+ 3195us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:15:04.205031 (+ 4622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:15:04.205044 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:15:04.205075 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:15:04.205827 (+ 752us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:15:04.206334 (+ 507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:15:04.206338 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:15:04.206341 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:15:04.206564 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:15:04.207539 (+ 975us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:15:04.207547 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:15:04.208043 (+ 496us) client_negotiation.cc:769] Sending connection context
0811 19:15:04.208374 (+ 331us) client_negotiation.cc:240] Negotiation successful
0811 19:15:04.209027 (+ 653us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":676,"thread_start_us":404,"threads_started":1}
I20260811 19:15:04.212713 27476 heartbeater.cc:347] Connected to a master server at 127.3.61.190:35949
I20260811 19:15:04.213110 27476 heartbeater.cc:464] Registering TS with master...
I20260811 19:15:04.214155 27476 heartbeater.cc:511] Master 127.3.61.190:35949 requested a full tablet report, sending...
I20260811 19:15:04.216214 27005 ts_manager.cc:194] Registered new tserver with Master: 40aa419517fc4e9db6457742b1257881 (127.3.61.130:38049)
I20260811 19:15:04.217110 27005 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.130@KRBTEST.COM'} at 127.3.61.130:40773
WARNING: no policy specified for kudu/127.3.61.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab.
Entry for principal kudu/127.3.61.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab.
I20260811 19:15:04.265558 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.131:0
--local_ip_for_outbound_sockets=127.3.61.131
--webserver_interface=127.3.61.131
--webserver_port=0
--tserver_master_addrs=127.3.61.190:35949
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab
--principal=kudu/127.3.61.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:42151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:15:04.602030 27280 heartbeater.cc:503] Master 127.3.61.190:35949 was elected leader, sending a full tablet report...
W20260811 19:15:04.654498 27484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:15:04.654834 27484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:15:04.654969 27484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:15:04.663101 27484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:15:04.663261 27484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:15:04.663419 27484 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.131
I20260811 19:15:04.676699 27484 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:42151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab
--principal=kudu/127.3.61.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.61.131
--webserver_port=0
--tserver_master_addrs=127.3.61.190:35949
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.131
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:15:04.678779 27484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:15:04.680850 27484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:15:04.696805 27489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:15:04.696846 27493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:15:04.696776 27490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:15:04.698282 27484 server_base.cc:1034] running on GCE node
I20260811 19:15:04.699365 27484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:15:04.700716 27484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:15:04.701951 27484 hybrid_clock.cc:648] HybridClock initialized: now 1786475704701903 us; error 37 us; skew 500 ppm
I20260811 19:15:04.702310 27484 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:15:04 dist-test-slave-t7g5 krb5kdc[26939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475704, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:15:04.705840 27484 init.cc:377] Logged in from keytab as kudu/127.3.61.131@KRBTEST.COM (short username kudu)
I20260811 19:15:04.707477 27484 webserver.cc:460] Webserver started at http://127.3.61.131:46849/ using document root <none> and password file <none>
I20260811 19:15:04.708334 27484 fs_manager.cc:359] Metadata directory not provided
I20260811 19:15:04.708496 27484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:15:04.708832 27484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:15:04.712186 27484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-2/data/instance:
uuid: "89a4e359e4fe45b6ac8c844075a3f655"
format_stamp: "Formatted at 2026-08-11 19:15:04 on dist-test-slave-t7g5"
server_key: "8ce64225e68545cb75a939b990445614"
server_key_iv: "e12c889d1853b7ff716d3fd7b98540cb"
server_key_version: "encryptionkey@0"
I20260811 19:15:04.713229 27484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-2/wal/instance:
uuid: "89a4e359e4fe45b6ac8c844075a3f655"
format_stamp: "Formatted at 2026-08-11 19:15:04 on dist-test-slave-t7g5"
server_key: "8ce64225e68545cb75a939b990445614"
server_key_iv: "e12c889d1853b7ff716d3fd7b98540cb"
server_key_version: "encryptionkey@0"
I20260811 19:15:04.721750 27484 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.003s sys 0.004s
I20260811 19:15:04.725386 27501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:15:04.726905 27484 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:15:04.727082 27484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-2/wal
uuid: "89a4e359e4fe45b6ac8c844075a3f655"
format_stamp: "Formatted at 2026-08-11 19:15:04 on dist-test-slave-t7g5"
server_key: "8ce64225e68545cb75a939b990445614"
server_key_iv: "e12c889d1853b7ff716d3fd7b98540cb"
server_key_version: "encryptionkey@0"
I20260811 19:15:04.727293 27484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:15:04.747220 27484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:15:04.751835 27484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:15:04.752127 27484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:15:04.753868 27484 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:15:04.755421 27484 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:15:04.755522 27484 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:15:04.755650 27484 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:15:04.755703 27484 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:15:04.793049 27484 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.131:42375
I20260811 19:15:04.793095 27673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.131:42375 every 8 connection(s)
I20260811 19:15:04.794947 27484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
I20260811 19:15:04.803161 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 27484
I20260811 19:15:04.803370 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-2/wal/instance
I20260811 19:15:04.803808 3318 external_mini_cluster.cc:1428] Setting key a6cc680fccaf6fe15f831393ba6e7c3e
Aug 11 19:15:04 dist-test-slave-t7g5 krb5kdc[26939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475704, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.131@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:15:04.819252 27676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:04.798045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:43507 (local address 127.3.61.190:35949)
0811 19:15:04.798622 (+ 577us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:04.798629 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:04.799616 (+ 987us) server_negotiation.cc:407] Connection header received
0811 19:15:04.801913 (+ 2297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:04.801919 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:04.802016 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:04.802116 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:15:04.803869 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:04.804809 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:04.807033 (+ 2224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:04.807633 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:04.807740 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:04.811728 (+ 3988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:15:04.811735 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:15:04.811739 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:15:04.811792 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:15:04.815186 (+ 3394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:04.816060 (+ 874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:04.816065 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:04.816067 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:04.816215 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:04.816646 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:04.816651 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:04.816653 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:04.816959 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:15:04.817090 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:04.818338 (+ 1248us) server_negotiation.cc:299] Negotiation successful
0811 19:15:04.818772 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":284,"threads_started":1}
I20260811 19:15:04.819864 27677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:04.798470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35949 (local address 127.3.61.131:43507)
0811 19:15:04.799379 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:04.799442 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 19:15:04.801492 (+ 2050us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:15:04.802397 (+ 905us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:15:04.802406 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:15:04.803015 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:15:04.803682 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:04.803703 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:04.804994 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:04.804999 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:04.806865 (+ 1866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:04.806881 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:04.807882 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:04.807888 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:04.808105 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:04.808869 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:15:04.808903 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:15:04.811392 (+ 2489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:15:04.815364 (+ 3972us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:15:04.815372 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:15:04.815389 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:15:04.815900 (+ 511us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:15:04.816301 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:15:04.816304 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:15:04.816306 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:15:04.816505 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:15:04.817108 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:15:04.817115 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:15:04.817503 (+ 388us) client_negotiation.cc:769] Sending connection context
0811 19:15:04.817830 (+ 327us) client_negotiation.cc:240] Negotiation successful
0811 19:15:04.818424 (+ 594us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":340,"threads_started":1}
I20260811 19:15:04.821250 27674 heartbeater.cc:347] Connected to a master server at 127.3.61.190:35949
I20260811 19:15:04.821683 27674 heartbeater.cc:464] Registering TS with master...
I20260811 19:15:04.822572 27674 heartbeater.cc:511] Master 127.3.61.190:35949 requested a full tablet report, sending...
I20260811 19:15:04.824105 27005 ts_manager.cc:194] Registered new tserver with Master: 89a4e359e4fe45b6ac8c844075a3f655 (127.3.61.131:42375)
I20260811 19:15:04.825255 27005 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.131@KRBTEST.COM'} at 127.3.61.131:43507
I20260811 19:15:04.833734 3318 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 19:15:04.859730 27676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:04.844604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40958 (local address 127.3.61.190:35949)
0811 19:15:04.844793 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:04.844798 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:04.845268 (+ 470us) server_negotiation.cc:407] Connection header received
0811 19:15:04.845414 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:04.845420 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:04.845507 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:04.845605 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:15:04.847283 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:04.848208 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:04.849692 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:04.850177 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:04.850290 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:04.853382 (+ 3092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:15:04.853388 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:15:04.853391 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:15:04.853430 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:15:04.856398 (+ 2968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:04.857399 (+ 1001us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:04.857404 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:04.857406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:04.857552 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:04.858085 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:04.858089 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:04.858092 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:04.858484 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:15:04.858707 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:04.859296 (+ 589us) server_negotiation.cc:299] Negotiation successful
0811 19:15:04.859494 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 19:15:04.864445 27005 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40958:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 19:15:04.871974 27005 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:15:04.910480 27696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:04.888833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39744 (local address 127.3.61.129:43955)
0811 19:15:04.900699 (+ 11866us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:04.900713 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:04.900757 (+ 44us) server_negotiation.cc:407] Connection header received
0811 19:15:04.900909 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:04.900918 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:04.902271 (+ 1353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:04.903099 (+ 828us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:15:04.904134 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:04.905316 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:04.908124 (+ 2808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:04.909239 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:04.909369 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:04.909636 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:04.909967 (+ 331us) server_negotiation.cc:299] Negotiation successful
0811 19:15:04.910156 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11610,"thread_start_us":11435,"threads_started":1}
I20260811 19:15:04.910521 27693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:04.888584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:43955 (local address 127.0.0.1:39744)
0811 19:15:04.899684 (+ 11100us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:04.899704 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 19:15:04.899844 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:15:04.902602 (+ 2758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:15:04.902611 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:15:04.902661 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:15:04.903552 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:04.903569 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:04.905630 (+ 2061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:04.905636 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:04.907950 (+ 2314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:04.907966 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:04.909487 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:04.909494 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:04.909736 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:04.909778 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 19:15:04.909899 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 19:15:04.910103 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10956,"mutex_wait_us":51,"thread_start_us":8810,"threads_started":1}
I20260811 19:15:04.911834 27695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:04.888185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.130:38049 (local address 127.0.0.1:33416)
0811 19:15:04.897991 (+ 9806us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:04.898056 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 19:15:04.898373 (+ 317us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:15:04.902775 (+ 4402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:15:04.902784 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:15:04.902808 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:15:04.903557 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:04.903569 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:04.905531 (+ 1962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:04.905537 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:04.908354 (+ 2817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:04.908368 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:04.910999 (+ 2631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:04.911006 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:04.911191 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:04.911225 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 19:15:04.911311 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 19:15:04.911446 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9385,"mutex_wait_us":714,"thread_start_us":11221,"threads_started":1}
I20260811 19:15:04.912674 27694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:04.888468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33416 (local address 127.3.61.130:38049)
0811 19:15:04.901927 (+ 13459us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:04.901935 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:04.901970 (+ 35us) server_negotiation.cc:407] Connection header received
0811 19:15:04.902089 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:04.902098 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:04.902277 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:04.902522 (+ 245us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:15:04.903805 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:04.905305 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:04.908529 (+ 3224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:04.910480 (+ 1951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:04.910629 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:04.910828 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:04.912175 (+ 1347us) server_negotiation.cc:299] Negotiation successful
0811 19:15:04.912374 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13313,"thread_start_us":13211,"threads_started":1}
I20260811 19:15:04.913687 27700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:04.901473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.131:42375 (local address 127.0.0.1:36338)
0811 19:15:04.903813 (+ 2340us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:04.903836 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 19:15:04.904116 (+ 280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:15:04.905034 (+ 918us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:15:04.905039 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:15:04.905064 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:15:04.905428 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:04.905441 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:04.907302 (+ 1861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:04.907308 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:04.909288 (+ 1980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:04.909304 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:04.912847 (+ 3543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:04.912854 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:04.913034 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:04.913070 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 19:15:04.913171 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 19:15:04.913319 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2205,"mutex_wait_us":2,"thread_start_us":1965,"threads_started":1}
I20260811 19:15:04.913817 27699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:04.900358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36338 (local address 127.3.61.131:42375)
0811 19:15:04.901445 (+ 1087us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:04.901457 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:04.904088 (+ 2631us) server_negotiation.cc:407] Connection header received
0811 19:15:04.904291 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:04.904299 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:04.904549 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:04.904753 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:15:04.905628 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:04.907152 (+ 1524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:04.909804 (+ 2652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:04.912593 (+ 2789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:04.912761 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:04.912992 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:04.913288 (+ 296us) server_negotiation.cc:299] Negotiation successful
0811 19:15:04.913468 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":817,"thread_start_us":497,"threads_started":1}
I20260811 19:15:04.915220 27186 tablet_service.cc:1467] Processing CreateTablet for tablet ecfae97e2e54404986a94153829acd65 (DEFAULT_TABLE table=test-table [id=6fc9b684a20d497b99c68338fae5f31c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:15:04.917140 27574 tablet_service.cc:1467] Processing CreateTablet for tablet ecfae97e2e54404986a94153829acd65 (DEFAULT_TABLE table=test-table [id=6fc9b684a20d497b99c68338fae5f31c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:15:04.917505 27380 tablet_service.cc:1467] Processing CreateTablet for tablet ecfae97e2e54404986a94153829acd65 (DEFAULT_TABLE table=test-table [id=6fc9b684a20d497b99c68338fae5f31c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:15:04.918164 27186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecfae97e2e54404986a94153829acd65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:15:04.918406 27574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecfae97e2e54404986a94153829acd65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:15:04.918872 27380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecfae97e2e54404986a94153829acd65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:15:04.930953 27703 tablet_bootstrap.cc:492] T ecfae97e2e54404986a94153829acd65 P 89a4e359e4fe45b6ac8c844075a3f655: Bootstrap starting.
I20260811 19:15:04.933714 27704 tablet_bootstrap.cc:492] T ecfae97e2e54404986a94153829acd65 P 40aa419517fc4e9db6457742b1257881: Bootstrap starting.
I20260811 19:15:04.935019 27705 tablet_bootstrap.cc:492] T ecfae97e2e54404986a94153829acd65 P c5948773534942afa0c14b4ef53aff64: Bootstrap starting.
I20260811 19:15:04.935135 27703 tablet_bootstrap.cc:654] T ecfae97e2e54404986a94153829acd65 P 89a4e359e4fe45b6ac8c844075a3f655: Neither blocks nor log segments found. Creating new log.
I20260811 19:15:04.936661 27703 log.cc:824] T ecfae97e2e54404986a94153829acd65 P 89a4e359e4fe45b6ac8c844075a3f655: Log is configured to *not* fsync() on all Append() calls
I20260811 19:15:04.938097 27704 tablet_bootstrap.cc:654] T ecfae97e2e54404986a94153829acd65 P 40aa419517fc4e9db6457742b1257881: Neither blocks nor log segments found. Creating new log.
I20260811 19:15:04.938263 27705 tablet_bootstrap.cc:654] T ecfae97e2e54404986a94153829acd65 P c5948773534942afa0c14b4ef53aff64: Neither blocks nor log segments found. Creating new log.
I20260811 19:15:04.939402 27704 log.cc:824] T ecfae97e2e54404986a94153829acd65 P 40aa419517fc4e9db6457742b1257881: Log is configured to *not* fsync() on all Append() calls
I20260811 19:15:04.939487 27705 log.cc:824] T ecfae97e2e54404986a94153829acd65 P c5948773534942afa0c14b4ef53aff64: Log is configured to *not* fsync() on all Append() calls
I20260811 19:15:04.940296 27703 tablet_bootstrap.cc:492] T ecfae97e2e54404986a94153829acd65 P 89a4e359e4fe45b6ac8c844075a3f655: No bootstrap required, opened a new log
I20260811 19:15:04.940564 27703 ts_tablet_manager.cc:1397] T ecfae97e2e54404986a94153829acd65 P 89a4e359e4fe45b6ac8c844075a3f655: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260811 19:15:04.942785 27704 tablet_bootstrap.cc:492] T ecfae97e2e54404986a94153829acd65 P 40aa419517fc4e9db6457742b1257881: No bootstrap required, opened a new log
I20260811 19:15:04.943126 27704 ts_tablet_manager.cc:1397] T ecfae97e2e54404986a94153829acd65 P 40aa419517fc4e9db6457742b1257881: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.003s
I20260811 19:15:04.943797 27705 tablet_bootstrap.cc:492] T ecfae97e2e54404986a94153829acd65 P c5948773534942afa0c14b4ef53aff64: No bootstrap required, opened a new log
I20260811 19:15:04.944052 27705 ts_tablet_manager.cc:1397] T ecfae97e2e54404986a94153829acd65 P c5948773534942afa0c14b4ef53aff64: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.002s
I20260811 19:15:04.952035 27705 raft_consensus.cc:348] T ecfae97e2e54404986a94153829acd65 P c5948773534942afa0c14b4ef53aff64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40aa419517fc4e9db6457742b1257881" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38049 } } peers { permanent_uuid: "c5948773534942afa0c14b4ef53aff64" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 43955 } } peers { permanent_uuid: "89a4e359e4fe45b6ac8c844075a3f655" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 42375 } }
I20260811 19:15:04.952618 27705 raft_consensus.cc:374] T ecfae97e2e54404986a94153829acd65 P c5948773534942afa0c14b4ef53aff64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:15:04.952869 27705 raft_consensus.cc:729] T ecfae97e2e54404986a94153829acd65 P c5948773534942afa0c14b4ef53aff64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5948773534942afa0c14b4ef53aff64, State: Initialized, Role: FOLLOWER
I20260811 19:15:04.952960 27703 raft_consensus.cc:348] T ecfae97e2e54404986a94153829acd65 P 89a4e359e4fe45b6ac8c844075a3f655 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40aa419517fc4e9db6457742b1257881" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38049 } } peers { permanent_uuid: "c5948773534942afa0c14b4ef53aff64" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 43955 } } peers { permanent_uuid: "89a4e359e4fe45b6ac8c844075a3f655" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 42375 } }
I20260811 19:15:04.953617 27703 raft_consensus.cc:374] T ecfae97e2e54404986a94153829acd65 P 89a4e359e4fe45b6ac8c844075a3f655 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:15:04.953783 27703 raft_consensus.cc:729] T ecfae97e2e54404986a94153829acd65 P 89a4e359e4fe45b6ac8c844075a3f655 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89a4e359e4fe45b6ac8c844075a3f655, State: Initialized, Role: FOLLOWER
I20260811 19:15:04.953805 27705 consensus_queue.cc:260] T ecfae97e2e54404986a94153829acd65 P c5948773534942afa0c14b4ef53aff64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40aa419517fc4e9db6457742b1257881" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38049 } } peers { permanent_uuid: "c5948773534942afa0c14b4ef53aff64" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 43955 } } peers { permanent_uuid: "89a4e359e4fe45b6ac8c844075a3f655" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 42375 } }
I20260811 19:15:04.954546 27703 consensus_queue.cc:260] T ecfae97e2e54404986a94153829acd65 P 89a4e359e4fe45b6ac8c844075a3f655 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40aa419517fc4e9db6457742b1257881" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38049 } } peers { permanent_uuid: "c5948773534942afa0c14b4ef53aff64" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 43955 } } peers { permanent_uuid: "89a4e359e4fe45b6ac8c844075a3f655" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 42375 } }
I20260811 19:15:04.955431 27704 raft_consensus.cc:348] T ecfae97e2e54404986a94153829acd65 P 40aa419517fc4e9db6457742b1257881 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40aa419517fc4e9db6457742b1257881" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38049 } } peers { permanent_uuid: "c5948773534942afa0c14b4ef53aff64" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 43955 } } peers { permanent_uuid: "89a4e359e4fe45b6ac8c844075a3f655" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 42375 } }
I20260811 19:15:04.956046 27704 raft_consensus.cc:374] T ecfae97e2e54404986a94153829acd65 P 40aa419517fc4e9db6457742b1257881 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:15:04.956153 27674 heartbeater.cc:503] Master 127.3.61.190:35949 was elected leader, sending a full tablet report...
I20260811 19:15:04.956204 27704 raft_consensus.cc:729] T ecfae97e2e54404986a94153829acd65 P 40aa419517fc4e9db6457742b1257881 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40aa419517fc4e9db6457742b1257881, State: Initialized, Role: FOLLOWER
I20260811 19:15:04.957226 27704 consensus_queue.cc:260] T ecfae97e2e54404986a94153829acd65 P 40aa419517fc4e9db6457742b1257881 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40aa419517fc4e9db6457742b1257881" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38049 } } peers { permanent_uuid: "c5948773534942afa0c14b4ef53aff64" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 43955 } } peers { permanent_uuid: "89a4e359e4fe45b6ac8c844075a3f655" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 42375 } }
I20260811 19:15:04.959107 27703 ts_tablet_manager.cc:1428] T ecfae97e2e54404986a94153829acd65 P 89a4e359e4fe45b6ac8c844075a3f655: Time spent starting tablet: real 0.018s user 0.011s sys 0.005s
I20260811 19:15:04.962458 27712 raft_consensus.cc:482] T ecfae97e2e54404986a94153829acd65 P 40aa419517fc4e9db6457742b1257881 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:15:04.963939 27705 ts_tablet_manager.cc:1428] T ecfae97e2e54404986a94153829acd65 P c5948773534942afa0c14b4ef53aff64: Time spent starting tablet: real 0.020s user 0.013s sys 0.007s
I20260811 19:15:04.964473 27476 heartbeater.cc:503] Master 127.3.61.190:35949 was elected leader, sending a full tablet report...
I20260811 19:15:04.964428 27712 raft_consensus.cc:504] T ecfae97e2e54404986a94153829acd65 P 40aa419517fc4e9db6457742b1257881 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40aa419517fc4e9db6457742b1257881" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38049 } } peers { permanent_uuid: "c5948773534942afa0c14b4ef53aff64" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 43955 } } peers { permanent_uuid: "89a4e359e4fe45b6ac8c844075a3f655" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 42375 } }
I20260811 19:15:04.964138 27704 ts_tablet_manager.cc:1428] T ecfae97e2e54404986a94153829acd65 P 40aa419517fc4e9db6457742b1257881: Time spent starting tablet: real 0.021s user 0.014s sys 0.003s
I20260811 19:15:04.969792 27712 leader_election.cc:288] T ecfae97e2e54404986a94153829acd65 P 40aa419517fc4e9db6457742b1257881 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c5948773534942afa0c14b4ef53aff64 (127.3.61.129:43955), 89a4e359e4fe45b6ac8c844075a3f655 (127.3.61.131:42375)
I20260811 19:15:04.977401 27696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:04.967419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:58947 (local address 127.3.61.129:43955)
0811 19:15:04.967790 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:04.967797 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:04.970910 (+ 3113us) server_negotiation.cc:407] Connection header received
0811 19:15:04.971118 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:04.971124 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:04.971201 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:04.971290 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:15:04.972277 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:04.973005 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:04.975169 (+ 2164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:04.976477 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:04.976652 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:04.976730 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:04.977010 (+ 280us) server_negotiation.cc:299] Negotiation successful
0811 19:15:04.977154 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205}
I20260811 19:15:04.977432 27719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:04.966849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:43955 (local address 127.3.61.130:58947)
0811 19:15:04.970767 (+ 3918us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:04.970790 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 19:15:04.970970 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:15:04.971313 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:15:04.971318 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:15:04.971345 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:15:04.972099 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:04.972115 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:04.973129 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:04.973133 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:04.974980 (+ 1847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:04.974996 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:04.976713 (+ 1717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:04.976718 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:04.976850 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:04.976877 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 19:15:04.976965 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 19:15:04.977068 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3744,"mutex_wait_us":39,"thread_start_us":3937,"threads_started":1}
I20260811 19:15:04.978391 27699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:04.968769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:45203 (local address 127.3.61.131:42375)
0811 19:15:04.968983 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:04.968988 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:04.971200 (+ 2212us) server_negotiation.cc:407] Connection header received
0811 19:15:04.971329 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:04.971334 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:04.971517 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:04.971636 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:15:04.972553 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:04.974111 (+ 1558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:04.976190 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:04.977299 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:04.977464 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:04.977533 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:04.978080 (+ 547us) server_negotiation.cc:299] Negotiation successful
0811 19:15:04.978215 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 19:15:04.978616 27716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:04.967587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.131:42375 (local address 127.3.61.130:45203)
0811 19:15:04.971081 (+ 3494us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:04.971103 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 19:15:04.971228 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:15:04.971673 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:15:04.971677 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:15:04.971698 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:15:04.972171 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:04.972179 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:04.974263 (+ 2084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:04.974272 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:04.975880 (+ 1608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:04.975893 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:04.977760 (+ 1867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:04.977768 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:04.977915 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:04.977939 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 19:15:04.978019 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 19:15:04.978136 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3383,"mutex_wait_us":41,"thread_start_us":2797,"threads_started":1}
I20260811 19:15:04.978675 27215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecfae97e2e54404986a94153829acd65" candidate_uuid: "40aa419517fc4e9db6457742b1257881" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5948773534942afa0c14b4ef53aff64" is_pre_election: true
I20260811 19:15:04.979359 27215 raft_consensus.cc:2454] T ecfae97e2e54404986a94153829acd65 P c5948773534942afa0c14b4ef53aff64 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40aa419517fc4e9db6457742b1257881 in term 0.
I20260811 19:15:04.979624 27609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecfae97e2e54404986a94153829acd65" candidate_uuid: "40aa419517fc4e9db6457742b1257881" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89a4e359e4fe45b6ac8c844075a3f655" is_pre_election: true
I20260811 19:15:04.980186 27609 raft_consensus.cc:2454] T ecfae97e2e54404986a94153829acd65 P 89a4e359e4fe45b6ac8c844075a3f655 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40aa419517fc4e9db6457742b1257881 in term 0.
I20260811 19:15:04.980336 27311 leader_election.cc:302] T ecfae97e2e54404986a94153829acd65 P 40aa419517fc4e9db6457742b1257881 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40aa419517fc4e9db6457742b1257881, c5948773534942afa0c14b4ef53aff64; no voters:
I20260811 19:15:04.981088 27712 raft_consensus.cc:2781] T ecfae97e2e54404986a94153829acd65 P 40aa419517fc4e9db6457742b1257881 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:15:04.981207 27712 raft_consensus.cc:482] T ecfae97e2e54404986a94153829acd65 P 40aa419517fc4e9db6457742b1257881 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:15:04.981316 27712 raft_consensus.cc:3037] T ecfae97e2e54404986a94153829acd65 P 40aa419517fc4e9db6457742b1257881 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:15:04.983552 27712 raft_consensus.cc:504] T ecfae97e2e54404986a94153829acd65 P 40aa419517fc4e9db6457742b1257881 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40aa419517fc4e9db6457742b1257881" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38049 } } peers { permanent_uuid: "c5948773534942afa0c14b4ef53aff64" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 43955 } } peers { permanent_uuid: "89a4e359e4fe45b6ac8c844075a3f655" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 42375 } }
I20260811 19:15:04.984208 27712 leader_election.cc:288] T ecfae97e2e54404986a94153829acd65 P 40aa419517fc4e9db6457742b1257881 [CANDIDATE]: Term 1 election: Requested vote from peers c5948773534942afa0c14b4ef53aff64 (127.3.61.129:43955), 89a4e359e4fe45b6ac8c844075a3f655 (127.3.61.131:42375)
I20260811 19:15:04.984946 27215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecfae97e2e54404986a94153829acd65" candidate_uuid: "40aa419517fc4e9db6457742b1257881" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5948773534942afa0c14b4ef53aff64"
I20260811 19:15:04.985147 27215 raft_consensus.cc:3037] T ecfae97e2e54404986a94153829acd65 P c5948773534942afa0c14b4ef53aff64 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:15:04.985220 27609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecfae97e2e54404986a94153829acd65" candidate_uuid: "40aa419517fc4e9db6457742b1257881" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89a4e359e4fe45b6ac8c844075a3f655"
I20260811 19:15:04.985442 27609 raft_consensus.cc:3037] T ecfae97e2e54404986a94153829acd65 P 89a4e359e4fe45b6ac8c844075a3f655 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:15:04.987497 27215 raft_consensus.cc:2454] T ecfae97e2e54404986a94153829acd65 P c5948773534942afa0c14b4ef53aff64 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40aa419517fc4e9db6457742b1257881 in term 1.
I20260811 19:15:04.987741 27609 raft_consensus.cc:2454] T ecfae97e2e54404986a94153829acd65 P 89a4e359e4fe45b6ac8c844075a3f655 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40aa419517fc4e9db6457742b1257881 in term 1.
I20260811 19:15:04.988034 27311 leader_election.cc:302] T ecfae97e2e54404986a94153829acd65 P 40aa419517fc4e9db6457742b1257881 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40aa419517fc4e9db6457742b1257881, c5948773534942afa0c14b4ef53aff64; no voters:
I20260811 19:15:04.988370 27712 raft_consensus.cc:2781] T ecfae97e2e54404986a94153829acd65 P 40aa419517fc4e9db6457742b1257881 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:15:04.988549 27712 raft_consensus.cc:686] T ecfae97e2e54404986a94153829acd65 P 40aa419517fc4e9db6457742b1257881 [term 1 LEADER]: Becoming Leader. State: Replica: 40aa419517fc4e9db6457742b1257881, State: Running, Role: LEADER
I20260811 19:15:04.989101 27712 consensus_queue.cc:237] T ecfae97e2e54404986a94153829acd65 P 40aa419517fc4e9db6457742b1257881 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40aa419517fc4e9db6457742b1257881" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38049 } } peers { permanent_uuid: "c5948773534942afa0c14b4ef53aff64" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 43955 } } peers { permanent_uuid: "89a4e359e4fe45b6ac8c844075a3f655" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 42375 } }
I20260811 19:15:04.994570 27005 catalog_manager.cc:5495] T ecfae97e2e54404986a94153829acd65 P 40aa419517fc4e9db6457742b1257881 reported cstate change: term changed from 0 to 1, leader changed from <none> to 40aa419517fc4e9db6457742b1257881 (127.3.61.130). New cstate: current_term: 1 leader_uuid: "40aa419517fc4e9db6457742b1257881" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40aa419517fc4e9db6457742b1257881" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c5948773534942afa0c14b4ef53aff64" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 43955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "89a4e359e4fe45b6ac8c844075a3f655" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 42375 } health_report { overall_health: UNKNOWN } } }
I20260811 19:15:05.016363 27694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:05.009882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33420 (local address 127.3.61.130:38049)
0811 19:15:05.010113 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:05.010118 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:05.010145 (+ 27us) server_negotiation.cc:407] Connection header received
0811 19:15:05.010565 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:05.010571 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:05.010702 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:05.010900 (+ 198us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:15:05.011432 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:05.012242 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:05.013769 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:05.014204 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:05.014314 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:05.015045 (+ 731us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:15:05.015615 (+ 570us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:15:05.015742 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:05.015976 (+ 234us) server_negotiation.cc:299] Negotiation successful
0811 19:15:05.016060 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 19:15:05.025414 27609 raft_consensus.cc:1261] T ecfae97e2e54404986a94153829acd65 P 89a4e359e4fe45b6ac8c844075a3f655 [term 1 FOLLOWER]: Refusing update from remote peer 40aa419517fc4e9db6457742b1257881: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:15:05.025548 27215 raft_consensus.cc:1261] T ecfae97e2e54404986a94153829acd65 P c5948773534942afa0c14b4ef53aff64 [term 1 FOLLOWER]: Refusing update from remote peer 40aa419517fc4e9db6457742b1257881: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:15:05.026599 27714 consensus_queue.cc:1035] T ecfae97e2e54404986a94153829acd65 P 40aa419517fc4e9db6457742b1257881 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89a4e359e4fe45b6ac8c844075a3f655" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 42375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:15:05.027034 27712 consensus_queue.cc:1035] T ecfae97e2e54404986a94153829acd65 P 40aa419517fc4e9db6457742b1257881 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5948773534942afa0c14b4ef53aff64" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 43955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:15:05.039639 27723 mvcc.cc:205] Tried to move back new op lower bound from 7317404487772758016 to 7317404487639736320. Current Snapshot: MvccSnapshot[applied={T|T < 7317404487772758016}]
I20260811 19:15:05.058291 27005 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40958:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:15:05.058568 27005 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40958:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:15:05.063452 27005 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd: Sending DeleteTablet for 3 replicas of tablet ecfae97e2e54404986a94153829acd65
I20260811 19:15:05.064934 27380 tablet_service.cc:1514] Processing DeleteTablet for tablet ecfae97e2e54404986a94153829acd65 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:15:05 UTC) from {username='kudu', principal='kudu/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:33416
I20260811 19:15:05.064927 27574 tablet_service.cc:1514] Processing DeleteTablet for tablet ecfae97e2e54404986a94153829acd65 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:15:05 UTC) from {username='kudu', principal='kudu/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:36338
I20260811 19:15:05.065094 27186 tablet_service.cc:1514] Processing DeleteTablet for tablet ecfae97e2e54404986a94153829acd65 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:15:05 UTC) from {username='kudu', principal='kudu/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:39744
I20260811 19:15:05.066319 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 27097
I20260811 19:15:05.075976 27738 tablet_replica.cc:330] T ecfae97e2e54404986a94153829acd65 P 40aa419517fc4e9db6457742b1257881: stopping tablet replica
I20260811 19:15:05.076133 27742 tablet_replica.cc:330] T ecfae97e2e54404986a94153829acd65 P c5948773534942afa0c14b4ef53aff64: stopping tablet replica
I20260811 19:15:05.076666 27742 raft_consensus.cc:2229] T ecfae97e2e54404986a94153829acd65 P c5948773534942afa0c14b4ef53aff64 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:15:05.077286 27742 raft_consensus.cc:2258] T ecfae97e2e54404986a94153829acd65 P c5948773534942afa0c14b4ef53aff64 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:15:05.077863 27738 raft_consensus.cc:2229] T ecfae97e2e54404986a94153829acd65 P 40aa419517fc4e9db6457742b1257881 [term 1 LEADER]: Raft consensus shutting down.
I20260811 19:15:05.078150 27739 tablet_replica.cc:330] T ecfae97e2e54404986a94153829acd65 P 89a4e359e4fe45b6ac8c844075a3f655: stopping tablet replica
I20260811 19:15:05.078540 27739 raft_consensus.cc:2229] T ecfae97e2e54404986a94153829acd65 P 89a4e359e4fe45b6ac8c844075a3f655 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:15:05.078891 27738 raft_consensus.cc:2258] T ecfae97e2e54404986a94153829acd65 P 40aa419517fc4e9db6457742b1257881 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:15:05.079049 27739 raft_consensus.cc:2258] T ecfae97e2e54404986a94153829acd65 P 89a4e359e4fe45b6ac8c844075a3f655 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:15:05.081323 27738 ts_tablet_manager.cc:1905] T ecfae97e2e54404986a94153829acd65 P 40aa419517fc4e9db6457742b1257881: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:15:05.081574 27739 ts_tablet_manager.cc:1905] T ecfae97e2e54404986a94153829acd65 P 89a4e359e4fe45b6ac8c844075a3f655: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:15:05.084697 27742 ts_tablet_manager.cc:1905] T ecfae97e2e54404986a94153829acd65 P c5948773534942afa0c14b4ef53aff64: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:15:05.089967 27739 ts_tablet_manager.cc:1918] T ecfae97e2e54404986a94153829acd65 P 89a4e359e4fe45b6ac8c844075a3f655: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:15:05.090219 27739 log.cc:1196] T ecfae97e2e54404986a94153829acd65 P 89a4e359e4fe45b6ac8c844075a3f655: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-2/wal/wals/ecfae97e2e54404986a94153829acd65
I20260811 19:15:05.090808 27739 ts_tablet_manager.cc:1939] T ecfae97e2e54404986a94153829acd65 P 89a4e359e4fe45b6ac8c844075a3f655: Deleting consensus metadata
Aug 11 19:15:05 dist-test-slave-t7g5 krb5kdc[26939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475702, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.129@KRBTEST.COM
I20260811 19:15:05.092849 26985 catalog_manager.cc:4841] TS 89a4e359e4fe45b6ac8c844075a3f655 (127.3.61.131:42375): tablet ecfae97e2e54404986a94153829acd65 (table test-table [id=6fc9b684a20d497b99c68338fae5f31c]) successfully deleted
I20260811 19:15:05.095335 27738 ts_tablet_manager.cc:1918] T ecfae97e2e54404986a94153829acd65 P 40aa419517fc4e9db6457742b1257881: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:15:05.095597 27738 log.cc:1196] T ecfae97e2e54404986a94153829acd65 P 40aa419517fc4e9db6457742b1257881: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-1/wal/wals/ecfae97e2e54404986a94153829acd65
I20260811 19:15:05.096199 27738 ts_tablet_manager.cc:1939] T ecfae97e2e54404986a94153829acd65 P 40aa419517fc4e9db6457742b1257881: Deleting consensus metadata
I20260811 19:15:05.098140 26985 catalog_manager.cc:4841] TS 40aa419517fc4e9db6457742b1257881 (127.3.61.130:38049): tablet ecfae97e2e54404986a94153829acd65 (table test-table [id=6fc9b684a20d497b99c68338fae5f31c]) successfully deleted
I20260811 19:15:05.098723 27742 ts_tablet_manager.cc:1918] T ecfae97e2e54404986a94153829acd65 P c5948773534942afa0c14b4ef53aff64: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:15:05.098868 27742 log.cc:1196] T ecfae97e2e54404986a94153829acd65 P c5948773534942afa0c14b4ef53aff64: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-0/wal/wals/ecfae97e2e54404986a94153829acd65
I20260811 19:15:05.099259 27742 ts_tablet_manager.cc:1939] T ecfae97e2e54404986a94153829acd65 P c5948773534942afa0c14b4ef53aff64: Deleting consensus metadata
I20260811 19:15:05.101090 26984 catalog_manager.cc:4841] TS c5948773534942afa0c14b4ef53aff64 (127.3.61.129:43955): tablet ecfae97e2e54404986a94153829acd65 (table test-table [id=6fc9b684a20d497b99c68338fae5f31c]) successfully deleted
I20260811 19:15:05.102169 27696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:05.067872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39754 (local address 127.3.61.129:43955)
0811 19:15:05.068080 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:05.068087 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:05.080631 (+ 12544us) server_negotiation.cc:407] Connection header received
0811 19:15:05.080919 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:05.080925 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:05.081189 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:05.081387 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:15:05.083761 (+ 2374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:05.084974 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:05.087249 (+ 2275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:05.087881 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:05.087990 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:05.094218 (+ 6228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:15:05.094244 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:15:05.094266 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:15:05.094358 (+ 92us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:15:05.098732 (+ 4374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:05.099760 (+ 1028us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:05.099769 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:05.099772 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:05.099955 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:05.100428 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:05.100433 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:05.100436 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:05.100927 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:15:05.101217 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:05.101585 (+ 368us) server_negotiation.cc:299] Negotiation successful
0811 19:15:05.101821 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 19:15:05.102792 27275 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:15:05.203485 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 27097
I20260811 19:15:05.228490 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 27289
Aug 11 19:15:05 dist-test-slave-t7g5 krb5kdc[26939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475702, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.130@KRBTEST.COM
I20260811 19:15:05.248943 27694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:05.229402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33430 (local address 127.3.61.130:38049)
0811 19:15:05.229583 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:05.229587 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:05.229611 (+ 24us) server_negotiation.cc:407] Connection header received
0811 19:15:05.229916 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:05.229923 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:05.230012 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:05.230144 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:15:05.231889 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:05.232903 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:05.234964 (+ 2061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:05.235497 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:05.235606 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:05.241343 (+ 5737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:15:05.241369 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:15:05.241396 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:15:05.241459 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:15:05.245143 (+ 3684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:05.246212 (+ 1069us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:05.246221 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:05.246223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:05.246421 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:05.246858 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:05.246862 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:05.246865 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:05.247298 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:15:05.247602 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:05.248175 (+ 573us) server_negotiation.cc:299] Negotiation successful
0811 19:15:05.248393 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 19:15:05.249936 27468 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:15:05.365458 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 27289
I20260811 19:15:05.392243 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 27484
Aug 11 19:15:05 dist-test-slave-t7g5 krb5kdc[26939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475702, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.131@KRBTEST.COM
I20260811 19:15:05.407109 27699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:05.393000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36342 (local address 127.3.61.131:42375)
0811 19:15:05.393207 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:05.393213 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:05.393233 (+ 20us) server_negotiation.cc:407] Connection header received
0811 19:15:05.393402 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:05.393406 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:05.393485 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:05.393581 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:15:05.394929 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:05.395687 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:05.396949 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:05.397336 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:05.397441 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:05.401629 (+ 4188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:15:05.401643 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:15:05.401689 (+ 46us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:15:05.401725 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:15:05.404437 (+ 2712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:05.405087 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:05.405094 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:05.405096 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:05.405235 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:05.405599 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:05.405603 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:05.405605 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:05.406005 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:15:05.406280 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:05.406572 (+ 292us) server_negotiation.cc:299] Negotiation successful
0811 19:15:05.406804 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 19:15:05.407717 27668 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:15:05.496666 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 27484
I20260811 19:15:05.521792 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 26959
I20260811 19:15:05.522944 27042 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:15:05.634652 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 26959
I20260811 19:15:05.666548 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:35949
--webserver_interface=127.3.61.190
--webserver_port=33277
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:42151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.61.190:35949
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:15:06.029520 27755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:15:06.029958 27755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:15:06.030107 27755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:15:06.038136 27755 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:15:06.038293 27755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:15:06.038514 27755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:15:06.038610 27755 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:15:06.038678 27755 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:15:06.049351 27755 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:42151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.3.61.190:35949
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:35949
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.61.190
--webserver_port=33277
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:15:06.050805 27755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:15:06.052172 27755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:15:06.081928 27767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:15:06.081959 27768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:15:06.082170 27772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:15:06.083341 27755 server_base.cc:1034] running on GCE node
I20260811 19:15:06.084833 27755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:15:06.087406 27755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:15:06.088744 27755 hybrid_clock.cc:648] HybridClock initialized: now 1786475706088647 us; error 85 us; skew 500 ppm
I20260811 19:15:06.089332 27755 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:15:06 dist-test-slave-t7g5 krb5kdc[26939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475706, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:15:06.093313 27755 init.cc:377] Logged in from keytab as kudu/127.3.61.190@KRBTEST.COM (short username kudu)
I20260811 19:15:06.095534 27755 webserver.cc:460] Webserver started at http://127.3.61.190:33277/ using document root <none> and password file <none>
I20260811 19:15:06.096414 27755 fs_manager.cc:359] Metadata directory not provided
I20260811 19:15:06.096562 27755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:15:06.103835 27755 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.001s sys 0.003s
I20260811 19:15:06.108412 27783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:15:06.110255 27755 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 19:15:06.110474 27755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/master-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/master-0/wal
uuid: "053ea5d60afa4ce4a2e5e9d1d6ee97fd"
format_stamp: "Formatted at 2026-08-11 19:15:02 on dist-test-slave-t7g5"
server_key: "9486a0c26b1a1ebd6229783aad0bb6db"
server_key_iv: "f0c51706b27670ff1cf0a4a7ee1778a5"
server_key_version: "encryptionkey@0"
I20260811 19:15:06.111658 27755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:15:06.129211 27755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:15:06.134236 27755 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:15:06.134536 27755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:15:06.159988 27755 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.190:35949
I20260811 19:15:06.160066 27870 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.190:35949 every 8 connection(s)
I20260811 19:15:06.161842 27755 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
I20260811 19:15:06.165350 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 27755
I20260811 19:15:06.166411 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.129:43955
--local_ip_for_outbound_sockets=127.3.61.129
--tserver_master_addrs=127.3.61.190:35949
--webserver_port=45241
--webserver_interface=127.3.61.129
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab
--principal=kudu/127.3.61.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:42151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:15:06.169973 27873 sys_catalog.cc:263] Verifying existing consensus state
I20260811 19:15:06.175030 27873 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd: Bootstrap starting.
I20260811 19:15:06.203145 27873 log.cc:824] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd: Log is configured to *not* fsync() on all Append() calls
I20260811 19:15:06.218818 27873 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 19:15:06.219553 27873 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd: Bootstrap complete.
I20260811 19:15:06.230269 27873 raft_consensus.cc:348] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "053ea5d60afa4ce4a2e5e9d1d6ee97fd" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35949 } }
I20260811 19:15:06.231246 27873 raft_consensus.cc:729] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 053ea5d60afa4ce4a2e5e9d1d6ee97fd, State: Initialized, Role: FOLLOWER
I20260811 19:15:06.231927 27873 consensus_queue.cc:260] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "053ea5d60afa4ce4a2e5e9d1d6ee97fd" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35949 } }
I20260811 19:15:06.232437 27873 raft_consensus.cc:388] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:15:06.232679 27873 raft_consensus.cc:482] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:15:06.232904 27873 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd [term 1 FOLLOWER]: Advancing to term 2
I20260811 19:15:06.238864 27873 raft_consensus.cc:504] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "053ea5d60afa4ce4a2e5e9d1d6ee97fd" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35949 } }
I20260811 19:15:06.239794 27873 leader_election.cc:302] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 053ea5d60afa4ce4a2e5e9d1d6ee97fd; no voters:
I20260811 19:15:06.240607 27873 leader_election.cc:288] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 19:15:06.240906 27878 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd [term 2 FOLLOWER]: Leader election won for term 2
I20260811 19:15:06.242481 27878 raft_consensus.cc:686] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd [term 2 LEADER]: Becoming Leader. State: Replica: 053ea5d60afa4ce4a2e5e9d1d6ee97fd, State: Running, Role: LEADER
I20260811 19:15:06.243156 27878 consensus_queue.cc:237] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "053ea5d60afa4ce4a2e5e9d1d6ee97fd" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35949 } }
I20260811 19:15:06.243557 27873 sys_catalog.cc:564] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:15:06.246107 27881 sys_catalog.cc:455] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 053ea5d60afa4ce4a2e5e9d1d6ee97fd. Latest consensus state: current_term: 2 leader_uuid: "053ea5d60afa4ce4a2e5e9d1d6ee97fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "053ea5d60afa4ce4a2e5e9d1d6ee97fd" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35949 } } }
I20260811 19:15:06.246580 27881 sys_catalog.cc:458] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd [sys.catalog]: This master's current role is: LEADER
I20260811 19:15:06.246156 27880 sys_catalog.cc:455] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "053ea5d60afa4ce4a2e5e9d1d6ee97fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "053ea5d60afa4ce4a2e5e9d1d6ee97fd" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35949 } } }
I20260811 19:15:06.246735 27880 sys_catalog.cc:458] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd [sys.catalog]: This master's current role is: LEADER
I20260811 19:15:06.247617 27889 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:15:06.269095 27889 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:15:06.271034 27889 catalog_manager.cc:1259] Loaded cluster ID: c718e517bf764f458269c35efd63348a
I20260811 19:15:06.271281 27889 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:15:06.275820 27889 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260811 19:15:06.294006 27889 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260811 19:15:06.305197 27889 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:15:06.307118 27889 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd: Loaded TSK: 0
I20260811 19:15:06.308739 27889 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 19:15:06.572633 27875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:15:06.572957 27875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:15:06.573055 27875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:15:06.580665 27875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:15:06.580796 27875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:15:06.580942 27875 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.129
I20260811 19:15:06.591228 27875 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:42151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab
--principal=kudu/127.3.61.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.129:43955
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.61.129
--webserver_port=45241
--tserver_master_addrs=127.3.61.190:35949
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.129
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:15:06.592687 27875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:15:06.594094 27875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:15:06.610124 27912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:15:06.610105 27913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:15:06.610281 27919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:15:06.615803 27875 server_base.cc:1034] running on GCE node
I20260811 19:15:06.616681 27875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:15:06.618014 27875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:15:06.619297 27875 hybrid_clock.cc:648] HybridClock initialized: now 1786475706619212 us; error 78 us; skew 500 ppm
I20260811 19:15:06.619776 27875 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:15:06 dist-test-slave-t7g5 krb5kdc[26939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475706, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:15:06.623080 27875 init.cc:377] Logged in from keytab as kudu/127.3.61.129@KRBTEST.COM (short username kudu)
I20260811 19:15:06.624826 27875 webserver.cc:460] Webserver started at http://127.3.61.129:45241/ using document root <none> and password file <none>
I20260811 19:15:06.625597 27875 fs_manager.cc:359] Metadata directory not provided
I20260811 19:15:06.625770 27875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:15:06.631505 27875 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 19:15:06.634431 27927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:15:06.635928 27875 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:15:06.636101 27875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-0/wal
uuid: "c5948773534942afa0c14b4ef53aff64"
format_stamp: "Formatted at 2026-08-11 19:15:03 on dist-test-slave-t7g5"
server_key: "aa2e104d864826a3228fed8e38f88af1"
server_key_iv: "037da0fbd9387d24fcc99627c8a0165a"
server_key_version: "encryptionkey@0"
I20260811 19:15:06.637132 27875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:15:06.659983 27875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:15:06.664579 27875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:15:06.664876 27875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:15:06.666034 27875 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:15:06.667531 27875 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:15:06.667635 27875 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:15:06.667752 27875 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:15:06.667860 27875 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260811 19:15:06.706650 27875 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.129:43955
I20260811 19:15:06.706727 28108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.129:43955 every 8 connection(s)
I20260811 19:15:06.708227 27875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
I20260811 19:15:06.715746 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 27875
I20260811 19:15:06.716609 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.130:38049
--local_ip_for_outbound_sockets=127.3.61.130
--tserver_master_addrs=127.3.61.190:35949
--webserver_port=35339
--webserver_interface=127.3.61.130
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab
--principal=kudu/127.3.61.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:42151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 19:15:06 dist-test-slave-t7g5 krb5kdc[26939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475706, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.129@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:15:06.739841 28114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:06.711336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35949 (local address 127.3.61.129:44047)
0811 19:15:06.712121 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:06.712159 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 19:15:06.713910 (+ 1751us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:15:06.715502 (+ 1592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:15:06.715530 (+ 28us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:15:06.716423 (+ 893us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:15:06.717204 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:06.717233 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:06.719242 (+ 2009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:06.719250 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:06.721324 (+ 2074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:06.721357 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:06.722908 (+ 1551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:06.722917 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:06.723193 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:06.723878 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:15:06.723909 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:15:06.727259 (+ 3350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:15:06.733049 (+ 5790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:15:06.733064 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:15:06.733085 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:15:06.734051 (+ 966us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:15:06.734774 (+ 723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:15:06.734779 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:15:06.734781 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:15:06.735023 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:15:06.736303 (+ 1280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:15:06.736314 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:15:06.737130 (+ 816us) client_negotiation.cc:769] Sending connection context
0811 19:15:06.737496 (+ 366us) client_negotiation.cc:240] Negotiation successful
0811 19:15:06.738224 (+ 728us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":305,"threads_started":1}
I20260811 19:15:06.739877 28111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:06.711202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:44047 (local address 127.3.61.190:35949)
0811 19:15:06.711855 (+ 653us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:06.711872 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:06.712297 (+ 425us) server_negotiation.cc:407] Connection header received
0811 19:15:06.714466 (+ 2169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:06.714479 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:06.714813 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:06.715100 (+ 287us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:15:06.717508 (+ 2408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:06.719007 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:06.721548 (+ 2541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:06.722580 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:06.722789 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:06.727874 (+ 5085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:15:06.727889 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:15:06.727916 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:15:06.727984 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:15:06.732701 (+ 4717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:06.734294 (+ 1593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:06.734305 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:06.734315 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:06.734583 (+ 268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:06.735190 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:06.735195 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:06.735198 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:06.736008 (+ 810us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:15:06.736204 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:06.737891 (+ 1687us) server_negotiation.cc:299] Negotiation successful
0811 19:15:06.738434 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":260,"threads_started":1}
I20260811 19:15:06.743362 28109 heartbeater.cc:347] Connected to a master server at 127.3.61.190:35949
I20260811 19:15:06.743731 28109 heartbeater.cc:464] Registering TS with master...
I20260811 19:15:06.745100 28109 heartbeater.cc:511] Master 127.3.61.190:35949 requested a full tablet report, sending...
I20260811 19:15:06.748417 27813 ts_manager.cc:194] Registered new tserver with Master: c5948773534942afa0c14b4ef53aff64 (127.3.61.129:43955)
I20260811 19:15:06.750075 27813 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.129@KRBTEST.COM'} at 127.3.61.129:44047
W20260811 19:15:07.087311 28116 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:15:07.087635 28116 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:15:07.087764 28116 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:15:07.095435 28116 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:15:07.095605 28116 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:15:07.095753 28116 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.130
I20260811 19:15:07.106153 28116 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:42151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab
--principal=kudu/127.3.61.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.130:38049
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.61.130
--webserver_port=35339
--tserver_master_addrs=127.3.61.190:35949
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.130
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:15:07.107587 28116 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:15:07.108987 28116 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:15:07.124058 28125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:15:07.124109 28126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:15:07.124135 28128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:15:07.125530 28116 server_base.cc:1034] running on GCE node
I20260811 19:15:07.126550 28116 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:15:07.128002 28116 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:15:07.129339 28116 hybrid_clock.cc:648] HybridClock initialized: now 1786475707129255 us; error 56 us; skew 500 ppm
I20260811 19:15:07.129876 28116 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:15:07 dist-test-slave-t7g5 krb5kdc[26939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475707, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:15:07.133896 28116 init.cc:377] Logged in from keytab as kudu/127.3.61.130@KRBTEST.COM (short username kudu)
I20260811 19:15:07.135643 28116 webserver.cc:460] Webserver started at http://127.3.61.130:35339/ using document root <none> and password file <none>
I20260811 19:15:07.136431 28116 fs_manager.cc:359] Metadata directory not provided
I20260811 19:15:07.136574 28116 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:15:07.142669 28116 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 19:15:07.145893 28144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:15:07.147691 28116 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 19:15:07.147886 28116 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-1/wal
uuid: "40aa419517fc4e9db6457742b1257881"
format_stamp: "Formatted at 2026-08-11 19:15:04 on dist-test-slave-t7g5"
server_key: "184bba89543eb0ebf583f36b6d44c07d"
server_key_iv: "593a93631308d45ab515262433bcf256"
server_key_version: "encryptionkey@0"
I20260811 19:15:07.149072 28116 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:15:07.171218 28116 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:15:07.176383 28116 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:15:07.176692 28116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:15:07.177933 28116 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:15:07.179550 28116 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:15:07.179654 28116 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:15:07.179780 28116 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:15:07.179850 28116 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:15:07.217301 28116 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.130:38049
I20260811 19:15:07.217365 28341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.130:38049 every 8 connection(s)
I20260811 19:15:07.218873 28116 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
I20260811 19:15:07.227995 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 28116
I20260811 19:15:07.228624 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.131:42375
--local_ip_for_outbound_sockets=127.3.61.131
--tserver_master_addrs=127.3.61.190:35949
--webserver_port=46849
--webserver_interface=127.3.61.131
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab
--principal=kudu/127.3.61.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:42151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 19:15:07 dist-test-slave-t7g5 krb5kdc[26939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475707, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.130@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:15:07.246364 28111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:07.222347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:39103 (local address 127.3.61.190:35949)
0811 19:15:07.222586 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:07.222591 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:07.223831 (+ 1240us) server_negotiation.cc:407] Connection header received
0811 19:15:07.225773 (+ 1942us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:07.225779 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:07.225895 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:07.225986 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:15:07.227600 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:07.228360 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:07.230069 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:07.230605 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:07.230935 (+ 330us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:07.235390 (+ 4455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:15:07.235396 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:15:07.235400 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:15:07.235448 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:15:07.242321 (+ 6873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:07.243510 (+ 1189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:07.243516 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:07.243519 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:07.243699 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:07.244288 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:07.244292 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:07.244295 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:07.244673 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:15:07.244814 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:07.245879 (+ 1065us) server_negotiation.cc:299] Negotiation successful
0811 19:15:07.246104 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 19:15:07.247803 28345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:07.222824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35949 (local address 127.3.61.130:39103)
0811 19:15:07.223638 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:07.223689 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 19:15:07.225461 (+ 1772us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:15:07.226265 (+ 804us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:15:07.226273 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:15:07.226831 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:15:07.227421 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:07.227440 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:07.228522 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:07.228526 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:07.229915 (+ 1389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:07.229929 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:07.231168 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:07.231181 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:07.231516 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:07.232372 (+ 856us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:15:07.232408 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:15:07.234981 (+ 2573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:15:07.242527 (+ 7546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:15:07.242538 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:15:07.242554 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:15:07.243275 (+ 721us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:15:07.243825 (+ 550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:15:07.243831 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:15:07.243835 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:15:07.244122 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:15:07.244807 (+ 685us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:15:07.244816 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:15:07.245382 (+ 566us) client_negotiation.cc:769] Sending connection context
0811 19:15:07.245729 (+ 347us) client_negotiation.cc:240] Negotiation successful
0811 19:15:07.246313 (+ 584us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":335,"threads_started":1}
I20260811 19:15:07.250008 28343 heartbeater.cc:347] Connected to a master server at 127.3.61.190:35949
I20260811 19:15:07.250391 28343 heartbeater.cc:464] Registering TS with master...
I20260811 19:15:07.251344 28343 heartbeater.cc:511] Master 127.3.61.190:35949 requested a full tablet report, sending...
I20260811 19:15:07.253248 27813 ts_manager.cc:194] Registered new tserver with Master: 40aa419517fc4e9db6457742b1257881 (127.3.61.130:38049)
I20260811 19:15:07.254217 27813 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.130@KRBTEST.COM'} at 127.3.61.130:39103
W20260811 19:15:07.617946 28346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:15:07.618260 28346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:15:07.618386 28346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:15:07.626888 28346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:15:07.627028 28346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:15:07.627182 28346 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.131
I20260811 19:15:07.638137 28346 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:42151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab
--principal=kudu/127.3.61.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.131:42375
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.61.131
--webserver_port=46849
--tserver_master_addrs=127.3.61.190:35949
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.131
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:15:07.639557 28346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:15:07.641041 28346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:15:07.657361 28357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:15:07.657416 28354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:15:07.657967 28364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:15:07.658170 28346 server_base.cc:1034] running on GCE node
I20260811 19:15:07.659204 28346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:15:07.660583 28346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:15:07.661809 28346 hybrid_clock.cc:648] HybridClock initialized: now 1786475707661717 us; error 69 us; skew 500 ppm
I20260811 19:15:07.662269 28346 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:15:07 dist-test-slave-t7g5 krb5kdc[26939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475707, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:15:07.668576 28346 init.cc:377] Logged in from keytab as kudu/127.3.61.131@KRBTEST.COM (short username kudu)
I20260811 19:15:07.670437 28346 webserver.cc:460] Webserver started at http://127.3.61.131:46849/ using document root <none> and password file <none>
I20260811 19:15:07.671192 28346 fs_manager.cc:359] Metadata directory not provided
I20260811 19:15:07.671322 28346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:15:07.677690 28346 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260811 19:15:07.681629 28374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:15:07.683558 28346 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 19:15:07.683748 28346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-2/wal
uuid: "89a4e359e4fe45b6ac8c844075a3f655"
format_stamp: "Formatted at 2026-08-11 19:15:04 on dist-test-slave-t7g5"
server_key: "8ce64225e68545cb75a939b990445614"
server_key_iv: "e12c889d1853b7ff716d3fd7b98540cb"
server_key_version: "encryptionkey@0"
I20260811 19:15:07.684780 28346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:15:07.716852 28346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:15:07.721944 28346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:15:07.722263 28346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:15:07.723583 28346 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:15:07.725806 28346 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:15:07.725932 28346 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:15:07.726058 28346 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:15:07.726127 28346 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:15:07.754926 28109 heartbeater.cc:503] Master 127.3.61.190:35949 was elected leader, sending a full tablet report...
I20260811 19:15:07.765404 28346 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.131:42375
I20260811 19:15:07.765476 28544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.131:42375 every 8 connection(s)
I20260811 19:15:07.767017 28346 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
I20260811 19:15:07.769265 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 28346
Aug 11 19:15:07 dist-test-slave-t7g5 krb5kdc[26939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475707, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.131@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:15:07.787524 28550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:07.770689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40968 (local address 127.3.61.190:35949)
0811 19:15:07.772363 (+ 1674us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:07.772370 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:07.772396 (+ 26us) server_negotiation.cc:407] Connection header received
0811 19:15:07.772621 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:07.772633 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:07.772800 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:07.772975 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:15:07.774953 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:07.775770 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:07.777275 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:07.777723 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:07.777834 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:07.781047 (+ 3213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:15:07.781051 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:15:07.781055 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:15:07.781104 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:15:07.784558 (+ 3454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:07.785352 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:07.785357 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:07.785359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:07.785497 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:07.786008 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:07.786012 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:07.786015 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:07.786337 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:15:07.786649 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:07.786885 (+ 236us) server_negotiation.cc:299] Negotiation successful
0811 19:15:07.787136 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1509,"thread_start_us":1322,"threads_started":1}
I20260811 19:15:07.794770 28551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:07.772189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:33981 (local address 127.3.61.190:35949)
0811 19:15:07.773023 (+ 834us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:07.773030 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:07.774009 (+ 979us) server_negotiation.cc:407] Connection header received
0811 19:15:07.776744 (+ 2735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:07.776750 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:07.776846 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:07.776981 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:15:07.779102 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:07.780207 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:07.782659 (+ 2452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:07.783373 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:07.783525 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:07.787538 (+ 4013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:15:07.787544 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:15:07.787547 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:15:07.787577 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:15:07.790935 (+ 3358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:07.791865 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:07.791873 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:07.791875 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:07.792005 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:07.792525 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:07.792530 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:07.792532 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:07.792845 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:15:07.792981 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:07.794035 (+ 1054us) server_negotiation.cc:299] Negotiation successful
0811 19:15:07.794326 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":670,"thread_start_us":413,"threads_started":1}
I20260811 19:15:07.795666 28552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:07.772153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35949 (local address 127.3.61.131:33981)
0811 19:15:07.773786 (+ 1633us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:07.773849 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 19:15:07.776398 (+ 2549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:15:07.777380 (+ 982us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:15:07.777393 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:15:07.778105 (+ 712us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:15:07.778911 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:07.778936 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:07.780492 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:07.780527 (+ 35us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:07.782476 (+ 1949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:07.782495 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:07.783676 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:07.783683 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:07.783949 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:07.784574 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:15:07.784600 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:15:07.787262 (+ 2662us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:15:07.791167 (+ 3905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:15:07.791178 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:15:07.791198 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:15:07.791697 (+ 499us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:15:07.792145 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:15:07.792149 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:15:07.792152 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:15:07.792373 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:15:07.793009 (+ 636us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:15:07.793018 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:15:07.793402 (+ 384us) client_negotiation.cc:769] Sending connection context
0811 19:15:07.793726 (+ 324us) client_negotiation.cc:240] Negotiation successful
0811 19:15:07.794199 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1209,"thread_start_us":976,"threads_started":1}
I20260811 19:15:07.797734 28546 heartbeater.cc:347] Connected to a master server at 127.3.61.190:35949
I20260811 19:15:07.798120 28546 heartbeater.cc:464] Registering TS with master...
I20260811 19:15:07.798911 28546 heartbeater.cc:511] Master 127.3.61.190:35949 requested a full tablet report, sending...
I20260811 19:15:07.800529 27813 ts_manager.cc:194] Registered new tserver with Master: 89a4e359e4fe45b6ac8c844075a3f655 (127.3.61.131:42375)
I20260811 19:15:07.801357 27813 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.131@KRBTEST.COM'} at 127.3.61.131:33981
I20260811 19:15:07.804317 3318 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 19:15:07.812366 28551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:07.805486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40978 (local address 127.3.61.190:35949)
0811 19:15:07.805724 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:07.805729 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:07.806517 (+ 788us) server_negotiation.cc:407] Connection header received
0811 19:15:07.806695 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:07.806698 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:07.806838 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:07.806981 (+ 143us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:15:07.807511 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:07.808252 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:07.809934 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:07.810472 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:07.810580 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:07.811191 (+ 611us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:15:07.811770 (+ 579us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:15:07.811884 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:07.812042 (+ 158us) server_negotiation.cc:299] Negotiation successful
0811 19:15:07.812121 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 19:15:07.813397 27813 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:40978:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 19:15:07.822919 27813 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:15:07.859835 28568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:07.836927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.130:38049 (local address 127.0.0.1:33440)
0811 19:15:07.849390 (+ 12463us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:07.849451 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 19:15:07.849677 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:15:07.850541 (+ 864us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:15:07.850548 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:15:07.850579 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:15:07.851019 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:07.851034 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:07.853350 (+ 2316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:07.853357 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:07.856150 (+ 2793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:07.856169 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:07.858692 (+ 2523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:07.858699 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:07.858930 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:07.859003 (+ 73us) client_negotiation.cc:769] Sending connection context
0811 19:15:07.859152 (+ 149us) client_negotiation.cc:240] Negotiation successful
0811 19:15:07.859383 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12155,"thread_start_us":13210,"threads_started":1}
I20260811 19:15:07.860903 28571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:07.837315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33440 (local address 127.3.61.130:38049)
0811 19:15:07.848473 (+ 11158us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:07.848486 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:07.849945 (+ 1459us) server_negotiation.cc:407] Connection header received
0811 19:15:07.850062 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:07.850069 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:07.850317 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:07.850508 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:15:07.851255 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:07.853144 (+ 1889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:07.856388 (+ 3244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:07.858116 (+ 1728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:07.858263 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:07.858510 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:07.860303 (+ 1793us) server_negotiation.cc:299] Negotiation successful
0811 19:15:07.860533 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10896,"thread_start_us":11177,"threads_started":1}
I20260811 19:15:07.863163 28567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:07.837127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:43955 (local address 127.0.0.1:39758)
0811 19:15:07.850431 (+ 13304us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:07.850448 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 19:15:07.850569 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:15:07.851753 (+ 1184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:15:07.851759 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:15:07.851789 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:15:07.852185 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:07.852196 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:07.856015 (+ 3819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:07.856022 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:07.858318 (+ 2296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:07.858335 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:07.862319 (+ 3984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:07.862326 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:07.862542 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:07.862578 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 19:15:07.862673 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 19:15:07.862825 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13174,"thread_start_us":12372,"threads_started":1}
I20260811 19:15:07.863307 28573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:07.837729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39758 (local address 127.3.61.129:43955)
0811 19:15:07.848950 (+ 11221us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:07.848959 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:07.850700 (+ 1741us) server_negotiation.cc:407] Connection header received
0811 19:15:07.850807 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:07.850815 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:07.851124 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:07.851340 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:15:07.852488 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:07.855829 (+ 3341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:07.858579 (+ 2750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:07.862058 (+ 3479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:07.862219 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:07.862501 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:07.862731 (+ 230us) server_negotiation.cc:299] Negotiation successful
0811 19:15:07.862934 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11000,"thread_start_us":11988,"threads_started":1}
I20260811 19:15:07.868388 27999 tablet_service.cc:1467] Processing CreateTablet for tablet 3f89239dd6924c3085330f769d8f456a (DEFAULT_TABLE table=test-table [id=0231907b20e047d1a6dca4521bf1984b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:15:07.869064 28231 tablet_service.cc:1467] Processing CreateTablet for tablet 3f89239dd6924c3085330f769d8f456a (DEFAULT_TABLE table=test-table [id=0231907b20e047d1a6dca4521bf1984b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:15:07.870314 28231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f89239dd6924c3085330f769d8f456a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:15:07.870385 27999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f89239dd6924c3085330f769d8f456a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:15:07.871435 28568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:07.850878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.131:42375 (local address 127.0.0.1:36344)
0811 19:15:07.861572 (+ 10694us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:07.861593 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 19:15:07.861727 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:15:07.864057 (+ 2330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:15:07.864062 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:15:07.864089 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:15:07.864276 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:07.864286 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:07.866836 (+ 2550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:07.866842 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:07.868670 (+ 1828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:07.868686 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:07.870677 (+ 1991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:07.870683 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:07.870860 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:07.870895 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 19:15:07.871014 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 19:15:07.871159 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10574,"thread_start_us":14077,"threads_started":1}
I20260811 19:15:07.871758 28579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:07.851081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36344 (local address 127.3.61.131:42375)
0811 19:15:07.863387 (+ 12306us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:07.863396 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:07.863438 (+ 42us) server_negotiation.cc:407] Connection header received
0811 19:15:07.863552 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:07.863561 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:07.863856 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:07.864065 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:15:07.864694 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:07.866597 (+ 1903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:07.868901 (+ 2304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:07.870460 (+ 1559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:07.870605 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:07.870850 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:07.871105 (+ 255us) server_negotiation.cc:299] Negotiation successful
0811 19:15:07.871292 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12110,"thread_start_us":13220,"threads_started":1}
I20260811 19:15:07.879798 28450 tablet_service.cc:1467] Processing CreateTablet for tablet 3f89239dd6924c3085330f769d8f456a (DEFAULT_TABLE table=test-table [id=0231907b20e047d1a6dca4521bf1984b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:15:07.882458 28450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f89239dd6924c3085330f769d8f456a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:15:07.888175 28581 tablet_bootstrap.cc:492] T 3f89239dd6924c3085330f769d8f456a P 40aa419517fc4e9db6457742b1257881: Bootstrap starting.
I20260811 19:15:07.891429 28582 tablet_bootstrap.cc:492] T 3f89239dd6924c3085330f769d8f456a P c5948773534942afa0c14b4ef53aff64: Bootstrap starting.
I20260811 19:15:07.892697 28581 tablet_bootstrap.cc:654] T 3f89239dd6924c3085330f769d8f456a P 40aa419517fc4e9db6457742b1257881: Neither blocks nor log segments found. Creating new log.
I20260811 19:15:07.894089 28581 log.cc:824] T 3f89239dd6924c3085330f769d8f456a P 40aa419517fc4e9db6457742b1257881: Log is configured to *not* fsync() on all Append() calls
I20260811 19:15:07.897018 28582 tablet_bootstrap.cc:654] T 3f89239dd6924c3085330f769d8f456a P c5948773534942afa0c14b4ef53aff64: Neither blocks nor log segments found. Creating new log.
I20260811 19:15:07.898468 28582 log.cc:824] T 3f89239dd6924c3085330f769d8f456a P c5948773534942afa0c14b4ef53aff64: Log is configured to *not* fsync() on all Append() calls
I20260811 19:15:07.900067 28581 tablet_bootstrap.cc:492] T 3f89239dd6924c3085330f769d8f456a P 40aa419517fc4e9db6457742b1257881: No bootstrap required, opened a new log
I20260811 19:15:07.900388 28581 ts_tablet_manager.cc:1397] T 3f89239dd6924c3085330f769d8f456a P 40aa419517fc4e9db6457742b1257881: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.003s
I20260811 19:15:07.902225 28587 tablet_bootstrap.cc:492] T 3f89239dd6924c3085330f769d8f456a P 89a4e359e4fe45b6ac8c844075a3f655: Bootstrap starting.
I20260811 19:15:07.906584 28587 tablet_bootstrap.cc:654] T 3f89239dd6924c3085330f769d8f456a P 89a4e359e4fe45b6ac8c844075a3f655: Neither blocks nor log segments found. Creating new log.
I20260811 19:15:07.907953 28582 tablet_bootstrap.cc:492] T 3f89239dd6924c3085330f769d8f456a P c5948773534942afa0c14b4ef53aff64: No bootstrap required, opened a new log
I20260811 19:15:07.908284 28582 ts_tablet_manager.cc:1397] T 3f89239dd6924c3085330f769d8f456a P c5948773534942afa0c14b4ef53aff64: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.004s
I20260811 19:15:07.908845 28587 log.cc:824] T 3f89239dd6924c3085330f769d8f456a P 89a4e359e4fe45b6ac8c844075a3f655: Log is configured to *not* fsync() on all Append() calls
I20260811 19:15:07.909511 28581 raft_consensus.cc:348] T 3f89239dd6924c3085330f769d8f456a P 40aa419517fc4e9db6457742b1257881 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40aa419517fc4e9db6457742b1257881" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38049 } } peers { permanent_uuid: "c5948773534942afa0c14b4ef53aff64" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 43955 } } peers { permanent_uuid: "89a4e359e4fe45b6ac8c844075a3f655" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 42375 } }
I20260811 19:15:07.910045 28581 raft_consensus.cc:374] T 3f89239dd6924c3085330f769d8f456a P 40aa419517fc4e9db6457742b1257881 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:15:07.910288 28581 raft_consensus.cc:729] T 3f89239dd6924c3085330f769d8f456a P 40aa419517fc4e9db6457742b1257881 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40aa419517fc4e9db6457742b1257881, State: Initialized, Role: FOLLOWER
I20260811 19:15:07.911242 28581 consensus_queue.cc:260] T 3f89239dd6924c3085330f769d8f456a P 40aa419517fc4e9db6457742b1257881 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40aa419517fc4e9db6457742b1257881" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38049 } } peers { permanent_uuid: "c5948773534942afa0c14b4ef53aff64" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 43955 } } peers { permanent_uuid: "89a4e359e4fe45b6ac8c844075a3f655" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 42375 } }
I20260811 19:15:07.912596 28343 heartbeater.cc:503] Master 127.3.61.190:35949 was elected leader, sending a full tablet report...
I20260811 19:15:07.912906 28581 ts_tablet_manager.cc:1428] T 3f89239dd6924c3085330f769d8f456a P 40aa419517fc4e9db6457742b1257881: Time spent starting tablet: real 0.012s user 0.013s sys 0.001s
I20260811 19:15:07.920440 28587 tablet_bootstrap.cc:492] T 3f89239dd6924c3085330f769d8f456a P 89a4e359e4fe45b6ac8c844075a3f655: No bootstrap required, opened a new log
I20260811 19:15:07.921487 28582 raft_consensus.cc:348] T 3f89239dd6924c3085330f769d8f456a P c5948773534942afa0c14b4ef53aff64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40aa419517fc4e9db6457742b1257881" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38049 } } peers { permanent_uuid: "c5948773534942afa0c14b4ef53aff64" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 43955 } } peers { permanent_uuid: "89a4e359e4fe45b6ac8c844075a3f655" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 42375 } }
I20260811 19:15:07.922108 28582 raft_consensus.cc:374] T 3f89239dd6924c3085330f769d8f456a P c5948773534942afa0c14b4ef53aff64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:15:07.922223 28587 ts_tablet_manager.cc:1397] T 3f89239dd6924c3085330f769d8f456a P 89a4e359e4fe45b6ac8c844075a3f655: Time spent bootstrapping tablet: real 0.020s user 0.000s sys 0.010s
I20260811 19:15:07.922224 28582 raft_consensus.cc:729] T 3f89239dd6924c3085330f769d8f456a P c5948773534942afa0c14b4ef53aff64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5948773534942afa0c14b4ef53aff64, State: Initialized, Role: FOLLOWER
I20260811 19:15:07.922955 28582 consensus_queue.cc:260] T 3f89239dd6924c3085330f769d8f456a P c5948773534942afa0c14b4ef53aff64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40aa419517fc4e9db6457742b1257881" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38049 } } peers { permanent_uuid: "c5948773534942afa0c14b4ef53aff64" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 43955 } } peers { permanent_uuid: "89a4e359e4fe45b6ac8c844075a3f655" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 42375 } }
I20260811 19:15:07.937695 28582 ts_tablet_manager.cc:1428] T 3f89239dd6924c3085330f769d8f456a P c5948773534942afa0c14b4ef53aff64: Time spent starting tablet: real 0.029s user 0.027s sys 0.004s
I20260811 19:15:07.937528 28587 raft_consensus.cc:348] T 3f89239dd6924c3085330f769d8f456a P 89a4e359e4fe45b6ac8c844075a3f655 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40aa419517fc4e9db6457742b1257881" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38049 } } peers { permanent_uuid: "c5948773534942afa0c14b4ef53aff64" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 43955 } } peers { permanent_uuid: "89a4e359e4fe45b6ac8c844075a3f655" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 42375 } }
I20260811 19:15:07.938130 28587 raft_consensus.cc:374] T 3f89239dd6924c3085330f769d8f456a P 89a4e359e4fe45b6ac8c844075a3f655 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:15:07.938249 28587 raft_consensus.cc:729] T 3f89239dd6924c3085330f769d8f456a P 89a4e359e4fe45b6ac8c844075a3f655 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89a4e359e4fe45b6ac8c844075a3f655, State: Initialized, Role: FOLLOWER
I20260811 19:15:07.938971 28587 consensus_queue.cc:260] T 3f89239dd6924c3085330f769d8f456a P 89a4e359e4fe45b6ac8c844075a3f655 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40aa419517fc4e9db6457742b1257881" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38049 } } peers { permanent_uuid: "c5948773534942afa0c14b4ef53aff64" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 43955 } } peers { permanent_uuid: "89a4e359e4fe45b6ac8c844075a3f655" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 42375 } }
I20260811 19:15:07.940469 28546 heartbeater.cc:503] Master 127.3.61.190:35949 was elected leader, sending a full tablet report...
I20260811 19:15:07.941319 28587 ts_tablet_manager.cc:1428] T 3f89239dd6924c3085330f769d8f456a P 89a4e359e4fe45b6ac8c844075a3f655: Time spent starting tablet: real 0.019s user 0.017s sys 0.000s
W20260811 19:15:07.960870 28110 tablet.cc:2369] T 3f89239dd6924c3085330f769d8f456a P c5948773534942afa0c14b4ef53aff64: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 19:15:07.971043 28344 tablet.cc:2369] T 3f89239dd6924c3085330f769d8f456a P 40aa419517fc4e9db6457742b1257881: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:15:07.988903 28590 raft_consensus.cc:482] T 3f89239dd6924c3085330f769d8f456a P 40aa419517fc4e9db6457742b1257881 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:15:07.989321 28590 raft_consensus.cc:504] T 3f89239dd6924c3085330f769d8f456a P 40aa419517fc4e9db6457742b1257881 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40aa419517fc4e9db6457742b1257881" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38049 } } peers { permanent_uuid: "c5948773534942afa0c14b4ef53aff64" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 43955 } } peers { permanent_uuid: "89a4e359e4fe45b6ac8c844075a3f655" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 42375 } }
I20260811 19:15:07.991310 28590 leader_election.cc:288] T 3f89239dd6924c3085330f769d8f456a P 40aa419517fc4e9db6457742b1257881 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c5948773534942afa0c14b4ef53aff64 (127.3.61.129:43955), 89a4e359e4fe45b6ac8c844075a3f655 (127.3.61.131:42375)
I20260811 19:15:07.999140 28579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:07.991800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:42953 (local address 127.3.61.131:42375)
0811 19:15:07.992022 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:07.992030 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:07.992968 (+ 938us) server_negotiation.cc:407] Connection header received
0811 19:15:07.993128 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:07.993133 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:07.993193 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:07.993270 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:15:07.993891 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:07.994655 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:07.996874 (+ 2219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:07.998161 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:07.998327 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:07.998411 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:07.998758 (+ 347us) server_negotiation.cc:299] Negotiation successful
0811 19:15:07.998886 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 19:15:07.999352 28595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:07.991801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.131:42375 (local address 127.3.61.130:42953)
0811 19:15:07.992842 (+ 1041us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:07.992858 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 19:15:07.993001 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:15:07.993301 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:15:07.993306 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:15:07.993329 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:15:07.993742 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:07.993752 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:07.994864 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:07.994871 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:07.996699 (+ 1828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:07.996713 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:07.998462 (+ 1749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:07.998471 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:07.998602 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:07.998626 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 19:15:07.998711 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 19:15:07.998867 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":932,"mutex_wait_us":21,"thread_start_us":584,"threads_started":1}
I20260811 19:15:07.999958 28573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:07.991876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:60707 (local address 127.3.61.129:43955)
0811 19:15:07.992088 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:07.992095 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:07.992926 (+ 831us) server_negotiation.cc:407] Connection header received
0811 19:15:07.993118 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:07.993124 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:07.993192 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:07.993270 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:15:07.994186 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:07.995412 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:07.997496 (+ 2084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:07.998744 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:07.998899 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:07.998971 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:07.999613 (+ 642us) server_negotiation.cc:299] Negotiation successful
0811 19:15:07.999756 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 19:15:08.000046 28594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:07.991751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:43955 (local address 127.3.61.130:60707)
0811 19:15:07.992799 (+ 1048us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:07.992816 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 19:15:07.992967 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:15:07.993309 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:15:07.993313 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:15:07.993436 (+ 123us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:15:07.993897 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:07.993910 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:07.995618 (+ 1708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:07.995622 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:07.997241 (+ 1619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:07.997257 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:07.998965 (+ 1708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:07.998971 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:07.999428 (+ 457us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:07.999461 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 19:15:07.999561 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 19:15:07.999694 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":913,"spinlock_wait_cycles":43136,"thread_start_us":630,"threads_started":1}
I20260811 19:15:08.000489 28478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f89239dd6924c3085330f769d8f456a" candidate_uuid: "40aa419517fc4e9db6457742b1257881" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89a4e359e4fe45b6ac8c844075a3f655" is_pre_election: true
I20260811 19:15:08.001068 28478 raft_consensus.cc:2454] T 3f89239dd6924c3085330f769d8f456a P 89a4e359e4fe45b6ac8c844075a3f655 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40aa419517fc4e9db6457742b1257881 in term 0.
I20260811 19:15:08.001130 28034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f89239dd6924c3085330f769d8f456a" candidate_uuid: "40aa419517fc4e9db6457742b1257881" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5948773534942afa0c14b4ef53aff64" is_pre_election: true
I20260811 19:15:08.001708 28034 raft_consensus.cc:2454] T 3f89239dd6924c3085330f769d8f456a P c5948773534942afa0c14b4ef53aff64 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40aa419517fc4e9db6457742b1257881 in term 0.
I20260811 19:15:08.001874 28147 leader_election.cc:302] T 3f89239dd6924c3085330f769d8f456a P 40aa419517fc4e9db6457742b1257881 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40aa419517fc4e9db6457742b1257881, 89a4e359e4fe45b6ac8c844075a3f655; no voters:
I20260811 19:15:08.002566 28590 raft_consensus.cc:2781] T 3f89239dd6924c3085330f769d8f456a P 40aa419517fc4e9db6457742b1257881 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:15:08.002717 28590 raft_consensus.cc:482] T 3f89239dd6924c3085330f769d8f456a P 40aa419517fc4e9db6457742b1257881 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:15:08.002825 28590 raft_consensus.cc:3037] T 3f89239dd6924c3085330f769d8f456a P 40aa419517fc4e9db6457742b1257881 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:15:08.005261 28590 raft_consensus.cc:504] T 3f89239dd6924c3085330f769d8f456a P 40aa419517fc4e9db6457742b1257881 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40aa419517fc4e9db6457742b1257881" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38049 } } peers { permanent_uuid: "c5948773534942afa0c14b4ef53aff64" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 43955 } } peers { permanent_uuid: "89a4e359e4fe45b6ac8c844075a3f655" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 42375 } }
I20260811 19:15:08.005975 28590 leader_election.cc:288] T 3f89239dd6924c3085330f769d8f456a P 40aa419517fc4e9db6457742b1257881 [CANDIDATE]: Term 1 election: Requested vote from peers c5948773534942afa0c14b4ef53aff64 (127.3.61.129:43955), 89a4e359e4fe45b6ac8c844075a3f655 (127.3.61.131:42375)
I20260811 19:15:08.006646 28034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f89239dd6924c3085330f769d8f456a" candidate_uuid: "40aa419517fc4e9db6457742b1257881" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5948773534942afa0c14b4ef53aff64"
I20260811 19:15:08.006858 28034 raft_consensus.cc:3037] T 3f89239dd6924c3085330f769d8f456a P c5948773534942afa0c14b4ef53aff64 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:15:08.006912 28478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f89239dd6924c3085330f769d8f456a" candidate_uuid: "40aa419517fc4e9db6457742b1257881" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89a4e359e4fe45b6ac8c844075a3f655"
I20260811 19:15:08.007239 28478 raft_consensus.cc:3037] T 3f89239dd6924c3085330f769d8f456a P 89a4e359e4fe45b6ac8c844075a3f655 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:15:08.009214 28034 raft_consensus.cc:2454] T 3f89239dd6924c3085330f769d8f456a P c5948773534942afa0c14b4ef53aff64 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40aa419517fc4e9db6457742b1257881 in term 1.
I20260811 19:15:08.009903 28146 leader_election.cc:302] T 3f89239dd6924c3085330f769d8f456a P 40aa419517fc4e9db6457742b1257881 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40aa419517fc4e9db6457742b1257881, c5948773534942afa0c14b4ef53aff64; no voters:
I20260811 19:15:08.010296 28590 raft_consensus.cc:2781] T 3f89239dd6924c3085330f769d8f456a P 40aa419517fc4e9db6457742b1257881 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:15:08.010412 28478 raft_consensus.cc:2454] T 3f89239dd6924c3085330f769d8f456a P 89a4e359e4fe45b6ac8c844075a3f655 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40aa419517fc4e9db6457742b1257881 in term 1.
I20260811 19:15:08.010862 28590 raft_consensus.cc:686] T 3f89239dd6924c3085330f769d8f456a P 40aa419517fc4e9db6457742b1257881 [term 1 LEADER]: Becoming Leader. State: Replica: 40aa419517fc4e9db6457742b1257881, State: Running, Role: LEADER
I20260811 19:15:08.011385 28590 consensus_queue.cc:237] T 3f89239dd6924c3085330f769d8f456a P 40aa419517fc4e9db6457742b1257881 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40aa419517fc4e9db6457742b1257881" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38049 } } peers { permanent_uuid: "c5948773534942afa0c14b4ef53aff64" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 43955 } } peers { permanent_uuid: "89a4e359e4fe45b6ac8c844075a3f655" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 42375 } }
I20260811 19:15:08.017534 27812 catalog_manager.cc:5495] T 3f89239dd6924c3085330f769d8f456a P 40aa419517fc4e9db6457742b1257881 reported cstate change: term changed from 0 to 1, leader changed from <none> to 40aa419517fc4e9db6457742b1257881 (127.3.61.130). New cstate: current_term: 1 leader_uuid: "40aa419517fc4e9db6457742b1257881" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40aa419517fc4e9db6457742b1257881" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c5948773534942afa0c14b4ef53aff64" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 43955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "89a4e359e4fe45b6ac8c844075a3f655" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 42375 } health_report { overall_health: UNKNOWN } } }
W20260811 19:15:08.019094 28547 tablet.cc:2369] T 3f89239dd6924c3085330f769d8f456a P 89a4e359e4fe45b6ac8c844075a3f655: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:15:08.054834 28571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:08.049243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33454 (local address 127.3.61.130:38049)
0811 19:15:08.049484 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:08.049490 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:08.049510 (+ 20us) server_negotiation.cc:407] Connection header received
0811 19:15:08.049617 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:08.049620 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:08.049742 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:08.049877 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:15:08.050271 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:08.050989 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:08.052561 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:08.052986 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:08.053098 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:08.053628 (+ 530us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:15:08.054206 (+ 578us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:15:08.054338 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:08.054458 (+ 120us) server_negotiation.cc:299] Negotiation successful
0811 19:15:08.054546 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 19:15:08.065924 28478 raft_consensus.cc:1261] T 3f89239dd6924c3085330f769d8f456a P 89a4e359e4fe45b6ac8c844075a3f655 [term 1 FOLLOWER]: Refusing update from remote peer 40aa419517fc4e9db6457742b1257881: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:15:08.065924 28034 raft_consensus.cc:1261] T 3f89239dd6924c3085330f769d8f456a P c5948773534942afa0c14b4ef53aff64 [term 1 FOLLOWER]: Refusing update from remote peer 40aa419517fc4e9db6457742b1257881: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:15:08.067343 28596 consensus_queue.cc:1035] T 3f89239dd6924c3085330f769d8f456a P 40aa419517fc4e9db6457742b1257881 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89a4e359e4fe45b6ac8c844075a3f655" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 42375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:15:08.067934 28590 consensus_queue.cc:1035] T 3f89239dd6924c3085330f769d8f456a P 40aa419517fc4e9db6457742b1257881 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5948773534942afa0c14b4ef53aff64" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 43955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:15:08.080708 28598 mvcc.cc:205] Tried to move back new op lower bound from 7317404500225191936 to 7317404500019322880. Current Snapshot: MvccSnapshot[applied={T|T < 7317404500225191936}]
I20260811 19:15:08.103446 27812 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:40978:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:15:08.103757 27812 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:40978:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:15:08.107856 27812 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd: Sending DeleteTablet for 3 replicas of tablet 3f89239dd6924c3085330f769d8f456a
I20260811 19:15:08.109375 27999 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f89239dd6924c3085330f769d8f456a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:15:08 UTC) from {username='kudu', principal='kudu/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:39758
I20260811 19:15:08.109375 28231 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f89239dd6924c3085330f769d8f456a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:15:08 UTC) from {username='kudu', principal='kudu/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:33440
I20260811 19:15:08.109505 28450 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f89239dd6924c3085330f769d8f456a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:15:08 UTC) from {username='kudu', principal='kudu/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:36344
I20260811 19:15:08.110817 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 27875
I20260811 19:15:08.122213 28614 tablet_replica.cc:330] T 3f89239dd6924c3085330f769d8f456a P c5948773534942afa0c14b4ef53aff64: stopping tablet replica
I20260811 19:15:08.122975 28614 raft_consensus.cc:2229] T 3f89239dd6924c3085330f769d8f456a P c5948773534942afa0c14b4ef53aff64 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:15:08.123791 28614 raft_consensus.cc:2258] T 3f89239dd6924c3085330f769d8f456a P c5948773534942afa0c14b4ef53aff64 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:15:08.126330 28615 tablet_replica.cc:330] T 3f89239dd6924c3085330f769d8f456a P 40aa419517fc4e9db6457742b1257881: stopping tablet replica
I20260811 19:15:08.127566 28614 ts_tablet_manager.cc:1905] T 3f89239dd6924c3085330f769d8f456a P c5948773534942afa0c14b4ef53aff64: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:15:08.128146 28615 raft_consensus.cc:2229] T 3f89239dd6924c3085330f769d8f456a P 40aa419517fc4e9db6457742b1257881 [term 1 LEADER]: Raft consensus shutting down.
I20260811 19:15:08.129016 28615 raft_consensus.cc:2258] T 3f89239dd6924c3085330f769d8f456a P 40aa419517fc4e9db6457742b1257881 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:15:08.137962 28573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:08.112406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39766 (local address 127.3.61.129:43955)
0811 19:15:08.112619 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:08.112625 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:08.112646 (+ 21us) server_negotiation.cc:407] Connection header received
0811 19:15:08.112712 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:08.112718 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:08.112811 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:08.112915 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:15:08.115951 (+ 3036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:08.117924 (+ 1973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:08.121429 (+ 3505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:08.122355 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:08.122518 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:08.128936 (+ 6418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:15:08.128950 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:15:08.128968 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:15:08.129026 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:15:08.133548 (+ 4522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:08.134653 (+ 1105us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:08.134663 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:08.134667 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:08.134867 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:08.135343 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:08.135348 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:08.135351 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:08.135851 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:15:08.136228 (+ 377us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:08.137276 (+ 1048us) server_negotiation.cc:299] Negotiation successful
0811 19:15:08.137555 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 19:15:08.139292 28097 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:15:08.139356 28615 ts_tablet_manager.cc:1905] T 3f89239dd6924c3085330f769d8f456a P 40aa419517fc4e9db6457742b1257881: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:15:08.139979 28619 tablet_replica.cc:330] T 3f89239dd6924c3085330f769d8f456a P 89a4e359e4fe45b6ac8c844075a3f655: stopping tablet replica
I20260811 19:15:08.140497 28619 raft_consensus.cc:2229] T 3f89239dd6924c3085330f769d8f456a P 89a4e359e4fe45b6ac8c844075a3f655 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:15:08.141194 28619 raft_consensus.cc:2258] T 3f89239dd6924c3085330f769d8f456a P 89a4e359e4fe45b6ac8c844075a3f655 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:15:08.148487 28619 ts_tablet_manager.cc:1905] T 3f89239dd6924c3085330f769d8f456a P 89a4e359e4fe45b6ac8c844075a3f655: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:15:08.148922 28615 ts_tablet_manager.cc:1918] T 3f89239dd6924c3085330f769d8f456a P 40aa419517fc4e9db6457742b1257881: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:15:08.149214 28615 log.cc:1196] T 3f89239dd6924c3085330f769d8f456a P 40aa419517fc4e9db6457742b1257881: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-1/wal/wals/3f89239dd6924c3085330f769d8f456a
I20260811 19:15:08.150061 28615 ts_tablet_manager.cc:1939] T 3f89239dd6924c3085330f769d8f456a P 40aa419517fc4e9db6457742b1257881: Deleting consensus metadata
I20260811 19:15:08.152328 27790 catalog_manager.cc:4841] TS 40aa419517fc4e9db6457742b1257881 (127.3.61.130:38049): tablet 3f89239dd6924c3085330f769d8f456a (table test-table [id=0231907b20e047d1a6dca4521bf1984b]) successfully deleted
I20260811 19:15:08.157145 28619 ts_tablet_manager.cc:1918] T 3f89239dd6924c3085330f769d8f456a P 89a4e359e4fe45b6ac8c844075a3f655: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:15:08.157369 28619 log.cc:1196] T 3f89239dd6924c3085330f769d8f456a P 89a4e359e4fe45b6ac8c844075a3f655: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-2/wal/wals/3f89239dd6924c3085330f769d8f456a
I20260811 19:15:08.158015 28619 ts_tablet_manager.cc:1939] T 3f89239dd6924c3085330f769d8f456a P 89a4e359e4fe45b6ac8c844075a3f655: Deleting consensus metadata
I20260811 19:15:08.161127 27790 catalog_manager.cc:4841] TS 89a4e359e4fe45b6ac8c844075a3f655 (127.3.61.131:42375): tablet 3f89239dd6924c3085330f769d8f456a (table test-table [id=0231907b20e047d1a6dca4521bf1984b]) successfully deleted
I20260811 19:15:08.282989 28614 ts_tablet_manager.cc:1918] T 3f89239dd6924c3085330f769d8f456a P c5948773534942afa0c14b4ef53aff64: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:15:08.283257 28614 log.cc:1196] T 3f89239dd6924c3085330f769d8f456a P c5948773534942afa0c14b4ef53aff64: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-0/wal/wals/3f89239dd6924c3085330f769d8f456a
I20260811 19:15:08.283761 28614 ts_tablet_manager.cc:1939] T 3f89239dd6924c3085330f769d8f456a P c5948773534942afa0c14b4ef53aff64: Deleting consensus metadata
I20260811 19:15:08.285585 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 27875
I20260811 19:15:08.285542 27787 catalog_manager.cc:4841] TS c5948773534942afa0c14b4ef53aff64 (127.3.61.129:43955): tablet 3f89239dd6924c3085330f769d8f456a (table test-table [id=0231907b20e047d1a6dca4521bf1984b]) successfully deleted
I20260811 19:15:08.310803 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 28116
I20260811 19:15:08.331029 28571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:08.311829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33458 (local address 127.3.61.130:38049)
0811 19:15:08.312062 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:08.312069 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:08.312094 (+ 25us) server_negotiation.cc:407] Connection header received
0811 19:15:08.312185 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:08.312191 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:08.312287 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:08.312414 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:15:08.314093 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:08.315625 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:08.318584 (+ 2959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:08.319933 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:08.320081 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:08.322981 (+ 2900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:15:08.322999 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:15:08.323025 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:15:08.323072 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:15:08.327209 (+ 4137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:08.328156 (+ 947us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:08.328166 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:08.328170 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:08.328362 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:08.328826 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:08.328831 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:08.328835 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:08.329359 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:15:08.329740 (+ 381us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:08.330265 (+ 525us) server_negotiation.cc:299] Negotiation successful
0811 19:15:08.330516 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 19:15:08.332139 28331 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:15:08.461822 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 28116
I20260811 19:15:08.510504 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 28346
I20260811 19:15:08.533018 28622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:08.511478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36350 (local address 127.3.61.131:42375)
0811 19:15:08.512014 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:08.512021 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:08.512048 (+ 27us) server_negotiation.cc:407] Connection header received
0811 19:15:08.512121 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:08.512127 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:08.512210 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:08.512348 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:15:08.514694 (+ 2346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:08.515478 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:08.517325 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:08.517829 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:08.517941 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:08.525637 (+ 7696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:15:08.525809 (+ 172us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:15:08.525836 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:15:08.525915 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:15:08.528902 (+ 2987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:08.529897 (+ 995us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:08.529911 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:08.529913 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:08.530191 (+ 278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:08.530928 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:08.530934 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:08.530937 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:08.531419 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:15:08.531694 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:08.532230 (+ 536us) server_negotiation.cc:299] Negotiation successful
0811 19:15:08.532647 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":259,"threads_started":1}
I20260811 19:15:08.534204 28539 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:15:08.632494 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 28346
I20260811 19:15:08.664634 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 27755
I20260811 19:15:08.665977 27862 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:15:08.789022 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 27755
I20260811 19:15:08.820601 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:35949
--webserver_interface=127.3.61.190
--webserver_port=33277
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:42151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.61.190:35949
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:15:09.217031 28627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:15:09.217391 28627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:15:09.217535 28627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:15:09.225117 28627 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:15:09.225256 28627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:15:09.225339 28627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:15:09.225425 28627 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:15:09.225495 28627 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:15:09.236002 28627 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:42151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.61.190:35949
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:35949
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.61.190
--webserver_port=33277
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:15:09.237507 28627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:15:09.238901 28627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:15:09.252102 28644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:15:09.252064 28636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:15:09.252143 28634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:15:09.253401 28627 server_base.cc:1034] running on GCE node
I20260811 19:15:09.254362 28627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:15:09.255972 28627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:15:09.257339 28627 hybrid_clock.cc:648] HybridClock initialized: now 1786475709257224 us; error 60 us; skew 500 ppm
I20260811 19:15:09.258018 28627 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:15:09 dist-test-slave-t7g5 krb5kdc[26939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475709, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:15:09.261561 28627 init.cc:377] Logged in from keytab as kudu/127.3.61.190@KRBTEST.COM (short username kudu)
I20260811 19:15:09.263338 28627 webserver.cc:460] Webserver started at http://127.3.61.190:33277/ using document root <none> and password file <none>
I20260811 19:15:09.264073 28627 fs_manager.cc:359] Metadata directory not provided
I20260811 19:15:09.264184 28627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:15:09.270704 28627 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 19:15:09.274892 28653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:15:09.277062 28627 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.000s
I20260811 19:15:09.277280 28627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/master-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/master-0/wal
uuid: "053ea5d60afa4ce4a2e5e9d1d6ee97fd"
format_stamp: "Formatted at 2026-08-11 19:15:02 on dist-test-slave-t7g5"
server_key: "9486a0c26b1a1ebd6229783aad0bb6db"
server_key_iv: "f0c51706b27670ff1cf0a4a7ee1778a5"
server_key_version: "encryptionkey@0"
I20260811 19:15:09.278589 28627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:15:09.300345 28627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:15:09.306135 28627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:15:09.306397 28627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:15:09.328578 28627 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.190:35949
I20260811 19:15:09.328598 28723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.190:35949 every 8 connection(s)
I20260811 19:15:09.330197 28627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
I20260811 19:15:09.334829 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 28627
I20260811 19:15:09.335990 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.129:43955
--local_ip_for_outbound_sockets=127.3.61.129
--tserver_master_addrs=127.3.61.190:35949
--webserver_port=45241
--webserver_interface=127.3.61.129
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab
--principal=kudu/127.3.61.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:42151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:15:09.338563 28726 sys_catalog.cc:263] Verifying existing consensus state
I20260811 19:15:09.343113 28726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd: Bootstrap starting.
I20260811 19:15:09.368237 28726 log.cc:824] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd: Log is configured to *not* fsync() on all Append() calls
I20260811 19:15:09.389457 28726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 19:15:09.390123 28726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd: Bootstrap complete.
I20260811 19:15:09.400653 28726 raft_consensus.cc:348] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "053ea5d60afa4ce4a2e5e9d1d6ee97fd" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35949 } }
I20260811 19:15:09.401700 28726 raft_consensus.cc:729] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 053ea5d60afa4ce4a2e5e9d1d6ee97fd, State: Initialized, Role: FOLLOWER
I20260811 19:15:09.402421 28726 consensus_queue.cc:260] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "053ea5d60afa4ce4a2e5e9d1d6ee97fd" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35949 } }
I20260811 19:15:09.402684 28726 raft_consensus.cc:388] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:15:09.402802 28726 raft_consensus.cc:482] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:15:09.402976 28726 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd [term 2 FOLLOWER]: Advancing to term 3
I20260811 19:15:09.407225 28726 raft_consensus.cc:504] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "053ea5d60afa4ce4a2e5e9d1d6ee97fd" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35949 } }
I20260811 19:15:09.407814 28726 leader_election.cc:302] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 053ea5d60afa4ce4a2e5e9d1d6ee97fd; no voters:
I20260811 19:15:09.408499 28726 leader_election.cc:288] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 19:15:09.408644 28734 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd [term 3 FOLLOWER]: Leader election won for term 3
I20260811 19:15:09.409809 28734 raft_consensus.cc:686] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd [term 3 LEADER]: Becoming Leader. State: Replica: 053ea5d60afa4ce4a2e5e9d1d6ee97fd, State: Running, Role: LEADER
I20260811 19:15:09.410461 28734 consensus_queue.cc:237] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "053ea5d60afa4ce4a2e5e9d1d6ee97fd" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35949 } }
I20260811 19:15:09.410864 28726 sys_catalog.cc:564] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:15:09.414153 28735 sys_catalog.cc:455] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "053ea5d60afa4ce4a2e5e9d1d6ee97fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "053ea5d60afa4ce4a2e5e9d1d6ee97fd" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35949 } } }
I20260811 19:15:09.414765 28735 sys_catalog.cc:458] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd [sys.catalog]: This master's current role is: LEADER
I20260811 19:15:09.415303 28738 sys_catalog.cc:455] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 053ea5d60afa4ce4a2e5e9d1d6ee97fd. Latest consensus state: current_term: 3 leader_uuid: "053ea5d60afa4ce4a2e5e9d1d6ee97fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "053ea5d60afa4ce4a2e5e9d1d6ee97fd" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35949 } } }
I20260811 19:15:09.415730 28738 sys_catalog.cc:458] T 00000000000000000000000000000000 P 053ea5d60afa4ce4a2e5e9d1d6ee97fd [sys.catalog]: This master's current role is: LEADER
I20260811 19:15:09.427343 28748 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:15:09.437922 28748 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:15:09.439813 28748 catalog_manager.cc:1259] Loaded cluster ID: c718e517bf764f458269c35efd63348a
I20260811 19:15:09.439936 28748 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260811 19:15:09.442250 28748 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f35700ae9ad google::LogMessage::Fail() at ??:0
@ 0x7f35700b188a google::LogMessage::SendToLog() at ??:0
@ 0x7f35700ae4de google::LogMessage::Flush() at ??:0
@ 0x7f35700afa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f3585ae1512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f3585ade976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f3585b66610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f3585b66439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f3572cfa345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f3572cfd91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f3572cfd799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f3572cd8eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f3575fb26db start_thread at ??:0
@ 0x7f356ef4861f clone at ??:0
W20260811 19:15:09.769788 28732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:15:09.770145 28732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:15:09.770298 28732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:15:09.779755 28732 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:15:09.780040 28732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:15:09.780373 28732 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.129
I20260811 19:15:09.792352 28732 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:42151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab
--principal=kudu/127.3.61.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.129:43955
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.61.129
--webserver_port=45241
--tserver_master_addrs=127.3.61.190:35949
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.129
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:15:09.794193 28732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:15:09.795979 28732 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:15:09.810254 28795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:15:09.810496 28797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:15:09.810658 28799 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:15:09.812109 28732 server_base.cc:1034] running on GCE node
I20260811 19:15:09.813236 28732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:15:09.814891 28732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:15:09.816274 28732 hybrid_clock.cc:648] HybridClock initialized: now 1786475709816204 us; error 48 us; skew 500 ppm
I20260811 19:15:09.816843 28732 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:15:09 dist-test-slave-t7g5 krb5kdc[26939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475709, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:15:09.820972 28732 init.cc:377] Logged in from keytab as kudu/127.3.61.129@KRBTEST.COM (short username kudu)
I20260811 19:15:09.822597 28732 webserver.cc:460] Webserver started at http://127.3.61.129:45241/ using document root <none> and password file <none>
I20260811 19:15:09.823323 28732 fs_manager.cc:359] Metadata directory not provided
I20260811 19:15:09.823436 28732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:15:09.829259 28732 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 19:15:09.833534 28807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:15:09.835216 28732 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 19:15:09.835399 28732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-0/wal
uuid: "c5948773534942afa0c14b4ef53aff64"
format_stamp: "Formatted at 2026-08-11 19:15:03 on dist-test-slave-t7g5"
server_key: "aa2e104d864826a3228fed8e38f88af1"
server_key_iv: "037da0fbd9387d24fcc99627c8a0165a"
server_key_version: "encryptionkey@0"
I20260811 19:15:09.836542 28732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:15:09.857241 28732 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:15:09.862855 28732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:15:09.863137 28732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:15:09.864401 28732 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:15:09.865935 28732 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:15:09.866034 28732 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:15:09.866123 28732 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:15:09.866189 28732 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:15:09.906306 28732 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.129:43955
I20260811 19:15:09.906385 28982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.129:43955 every 8 connection(s)
I20260811 19:15:09.908004 28732 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
I20260811 19:15:09.909727 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 28732
I20260811 19:15:09.910553 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.130:38049
--local_ip_for_outbound_sockets=127.3.61.130
--tserver_master_addrs=127.3.61.190:35949
--webserver_port=35339
--webserver_interface=127.3.61.130
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab
--principal=kudu/127.3.61.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:42151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:15:09.914404 28986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:09.911599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35949 (local address 127.3.61.129:59773)
0811 19:15:09.912824 (+ 1225us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:09.913104 (+ 280us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":347,"threads_started":1}
W20260811 19:15:09.915262 28983 heartbeater.cc:644] Failed to heartbeat to 127.3.61.190:35949 (0 consecutive failures): Network error: Failed to ping master at 127.3.61.190:35949: Client connection negotiation failed: client connection to 127.3.61.190:35949: connect: Connection refused (error 111)
I20260811 19:15:09.916805 28986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:09.916108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35949 (local address 127.3.61.129:52733)
0811 19:15:09.916328 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:09.916570 (+ 242us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260811 19:15:09.918304 28986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:09.917813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35949 (local address 127.3.61.129:38335)
0811 19:15:09.917998 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:09.918127 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 19:15:09.918740 28983 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 19:15:10.300630 28987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:15:10.300978 28987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:15:10.301096 28987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:15:10.309330 28987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:15:10.309495 28987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:15:10.309623 28987 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.130
I20260811 19:15:10.320580 28987 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:42151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab
--principal=kudu/127.3.61.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.130:38049
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.61.130
--webserver_port=35339
--tserver_master_addrs=127.3.61.190:35949
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.130
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:15:10.322240 28987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:15:10.323642 28987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:15:10.338879 29002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:15:10.338945 28999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:15:10.339165 28998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:15:10.339988 28987 server_base.cc:1034] running on GCE node
I20260811 19:15:10.340899 28987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:15:10.342033 28987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:15:10.343272 28987 hybrid_clock.cc:648] HybridClock initialized: now 1786475710343208 us; error 50 us; skew 500 ppm
I20260811 19:15:10.343648 28987 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:15:10 dist-test-slave-t7g5 krb5kdc[26939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475710, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:15:10.347625 28987 init.cc:377] Logged in from keytab as kudu/127.3.61.130@KRBTEST.COM (short username kudu)
I20260811 19:15:10.349241 28987 webserver.cc:460] Webserver started at http://127.3.61.130:35339/ using document root <none> and password file <none>
I20260811 19:15:10.350059 28987 fs_manager.cc:359] Metadata directory not provided
I20260811 19:15:10.350181 28987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:15:10.356125 28987 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.002s
I20260811 19:15:10.359632 29016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:15:10.361420 28987 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 19:15:10.361608 28987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-1/wal
uuid: "40aa419517fc4e9db6457742b1257881"
format_stamp: "Formatted at 2026-08-11 19:15:04 on dist-test-slave-t7g5"
server_key: "184bba89543eb0ebf583f36b6d44c07d"
server_key_iv: "593a93631308d45ab515262433bcf256"
server_key_version: "encryptionkey@0"
I20260811 19:15:10.362687 28987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:15:10.384208 28987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:15:10.388991 28987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:15:10.389272 28987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:15:10.390424 28987 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:15:10.391897 28987 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:15:10.392001 28987 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:15:10.392109 28987 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:15:10.392176 28987 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:15:10.431658 28987 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.130:38049
I20260811 19:15:10.431730 29192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.130:38049 every 8 connection(s)
I20260811 19:15:10.433261 28987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
I20260811 19:15:10.438608 29195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:10.436583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35949 (local address 127.3.61.130:48251)
0811 19:15:10.437418 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:10.437642 (+ 224us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":323,"threads_started":1}
W20260811 19:15:10.439360 29193 heartbeater.cc:644] Failed to heartbeat to 127.3.61.190:35949 (0 consecutive failures): Network error: Failed to ping master at 127.3.61.190:35949: Client connection negotiation failed: client connection to 127.3.61.190:35949: connect: Connection refused (error 111)
I20260811 19:15:10.439881 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 28987
I20260811 19:15:10.440220 29195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:10.439876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35949 (local address 127.3.61.130:52177)
0811 19:15:10.440038 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:10.440108 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 19:15:10.440611 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.131:42375
--local_ip_for_outbound_sockets=127.3.61.131
--tserver_master_addrs=127.3.61.190:35949
--webserver_port=46849
--webserver_interface=127.3.61.131
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab
--principal=kudu/127.3.61.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:42151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:15:10.441226 29195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:10.440817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35949 (local address 127.3.61.130:34905)
0811 19:15:10.441021 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:10.441104 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
W20260811 19:15:10.441437 29193 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 19:15:10.821091 29196 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:15:10.821517 29196 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:15:10.821631 29196 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:15:10.831195 29196 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:15:10.831363 29196 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:15:10.831537 29196 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.131
I20260811 19:15:10.844473 29196 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:42151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab
--principal=kudu/127.3.61.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.131:42375
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.61.131
--webserver_port=46849
--tserver_master_addrs=127.3.61.190:35949
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.131
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:15:10.845943 29196 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:15:10.847357 29196 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:15:10.861107 29205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:15:10.861142 29203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:15:10.861189 29202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:15:10.862267 29196 server_base.cc:1034] running on GCE node
I20260811 19:15:10.863008 29196 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:15:10.864183 29196 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:15:10.865568 29196 hybrid_clock.cc:648] HybridClock initialized: now 1786475710865442 us; error 41 us; skew 500 ppm
I20260811 19:15:10.866003 29196 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:15:10 dist-test-slave-t7g5 krb5kdc[26939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475710, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:15:10.869498 29196 init.cc:377] Logged in from keytab as kudu/127.3.61.131@KRBTEST.COM (short username kudu)
I20260811 19:15:10.871160 29196 webserver.cc:460] Webserver started at http://127.3.61.131:46849/ using document root <none> and password file <none>
I20260811 19:15:10.872007 29196 fs_manager.cc:359] Metadata directory not provided
I20260811 19:15:10.872138 29196 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:15:10.878551 29196 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 19:15:10.881937 29216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:15:10.883966 29196 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260811 19:15:10.884248 29196 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-2/wal
uuid: "89a4e359e4fe45b6ac8c844075a3f655"
format_stamp: "Formatted at 2026-08-11 19:15:04 on dist-test-slave-t7g5"
server_key: "8ce64225e68545cb75a939b990445614"
server_key_iv: "e12c889d1853b7ff716d3fd7b98540cb"
server_key_version: "encryptionkey@0"
I20260811 19:15:10.885571 29196 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:15:10.901736 29196 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:15:10.906365 29196 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:15:10.906644 29196 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:15:10.907732 29196 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:15:10.909133 29196 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:15:10.909230 29196 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:15:10.909341 29196 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:15:10.909412 29196 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:15:10.920787 29260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:10.919696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35949 (local address 127.3.61.129:36649)
0811 19:15:10.920304 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:10.920452 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":305,"threads_started":1}
I20260811 19:15:10.948514 29196 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.131:42375
I20260811 19:15:10.948593 29388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.131:42375 every 8 connection(s)
I20260811 19:15:10.950120 29196 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
I20260811 19:15:10.955065 29391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:10.953146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35949 (local address 127.3.61.131:34381)
0811 19:15:10.953978 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:10.954193 (+ 215us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":354,"threads_started":1}
I20260811 19:15:10.955729 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 29196
W20260811 19:15:10.955850 29389 heartbeater.cc:644] Failed to heartbeat to 127.3.61.190:35949 (0 consecutive failures): Network error: Failed to ping master at 127.3.61.190:35949: Client connection negotiation failed: client connection to 127.3.61.190:35949: connect: Connection refused (error 111)
I20260811 19:15:10.956825 29391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:10.956476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35949 (local address 127.3.61.131:58885)
0811 19:15:10.956638 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:10.956708 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 19:15:10.957763 29391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:10.957367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35949 (local address 127.3.61.131:33371)
0811 19:15:10.957511 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:10.957604 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 19:15:10.958019 29389 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 19:15:11.443063 29395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:11.442149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35949 (local address 127.3.61.130:52493)
0811 19:15:11.442685 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:11.442810 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":275,"threads_started":1}
I20260811 19:15:11.923275 29396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:11.922101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35949 (local address 127.3.61.129:33635)
0811 19:15:11.922866 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:11.922994 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":615,"thread_start_us":416,"threads_started":1}
I20260811 19:15:11.959988 29397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:11.958831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35949 (local address 127.3.61.131:39443)
0811 19:15:11.959527 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:11.959667 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":333,"threads_started":1}
I20260811 19:15:12.445056 29399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:12.444205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35949 (local address 127.3.61.130:34197)
0811 19:15:12.444767 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:12.444862 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":295,"threads_started":1}
I20260811 19:15:12.925746 29400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:12.924780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35949 (local address 127.3.61.129:46539)
0811 19:15:12.925367 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:12.925494 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":311,"threads_started":1}
I20260811 19:15:12.962234 29401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:12.961296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35949 (local address 127.3.61.131:60645)
0811 19:15:12.961926 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:12.962023 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":339,"threads_started":1}
I20260811 19:15:13.446895 29404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:13.446109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35949 (local address 127.3.61.130:55965)
0811 19:15:13.446616 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:13.446704 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":277,"threads_started":1}
I20260811 19:15:13.927467 29405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:13.926674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35949 (local address 127.3.61.129:38509)
0811 19:15:13.927178 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:13.927273 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":258,"threads_started":1}
I20260811 19:15:13.964298 29406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:13.963269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35949 (local address 127.3.61.131:33143)
0811 19:15:13.963942 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:13.964059 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":354,"threads_started":1}
I20260811 19:15:14.448559 29407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:14.447818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35949 (local address 127.3.61.130:45301)
0811 19:15:14.448298 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:14.448379 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":260,"threads_started":1}
I20260811 19:15:14.929436 29409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:14.928631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35949 (local address 127.3.61.129:45669)
0811 19:15:14.929162 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:14.929253 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":280,"threads_started":1}
I20260811 19:15:14.966473 29411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:14.965532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35949 (local address 127.3.61.131:58445)
0811 19:15:14.966104 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:14.966218 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":297,"threads_started":1}
I20260811 19:15:15.450618 29412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:15.449730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35949 (local address 127.3.61.130:35835)
0811 19:15:15.450336 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:15.450434 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":259,"threads_started":1}
I20260811 19:15:15.931346 29413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:15.930507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35949 (local address 127.3.61.129:39473)
0811 19:15:15.931060 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:15.931167 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":289,"threads_started":1}
I20260811 19:15:15.968407 29414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:15.967602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35949 (local address 127.3.61.131:40227)
0811 19:15:15.968138 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:15.968230 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":278,"threads_started":1}
I20260811 19:15:16.452694 29415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:16.451779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35949 (local address 127.3.61.130:55177)
0811 19:15:16.452376 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:16.452504 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":317,"threads_started":1}
I20260811 19:15:16.934252 29416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:16.932937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35949 (local address 127.3.61.129:50675)
0811 19:15:16.933879 (+ 942us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:16.934002 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":773,"thread_start_us":407,"threads_started":1}
I20260811 19:15:16.970194 29419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:16.969346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35949 (local address 127.3.61.131:58863)
0811 19:15:16.969912 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:16.970012 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":327,"threads_started":1}
I20260811 19:15:17.455319 29421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:17.454036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35949 (local address 127.3.61.130:34953)
0811 19:15:17.454926 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:17.455032 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":758,"thread_start_us":471,"threads_started":1}
I20260811 19:15:17.936955 29422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:17.935612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35949 (local address 127.3.61.129:42597)
0811 19:15:17.936395 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:17.936600 (+ 205us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":295,"threads_started":1}
I20260811 19:15:17.975371 29423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:17.971761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35949 (local address 127.3.61.131:58943)
0811 19:15:17.974975 (+ 3214us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:17.975083 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2593,"thread_start_us":2390,"threads_started":1}
I20260811 19:15:18.458232 29427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:18.457000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35949 (local address 127.3.61.130:40329)
0811 19:15:18.457802 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:18.457930 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":614,"thread_start_us":422,"threads_started":1}
I20260811 19:15:18.939759 29428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:18.938658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35949 (local address 127.3.61.129:41001)
0811 19:15:18.939406 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:18.939519 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":596,"thread_start_us":382,"threads_started":1}
I20260811 19:15:18.978084 29429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:18.976972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35949 (local address 127.3.61.131:56973)
0811 19:15:18.977715 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:18.977840 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":343,"threads_started":1}
I20260811 19:15:19.460815 29430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:19.459619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35949 (local address 127.3.61.130:51091)
0811 19:15:19.460420 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:19.460561 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":631,"thread_start_us":426,"threads_started":1}
I20260811 19:15:19.942199 29431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:19.941198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35949 (local address 127.3.61.129:53403)
0811 19:15:19.941878 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:19.942003 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":347,"threads_started":1}
I20260811 19:15:19.980569 29434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:19.979342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35949 (local address 127.3.61.131:48371)
0811 19:15:19.980162 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:19.980303 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":631,"thread_start_us":402,"threads_started":1}
I20260811 19:15:20.463519 29435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:20.462347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35949 (local address 127.3.61.130:54251)
0811 19:15:20.463083 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:20.463256 (+ 173us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":328,"threads_started":1}
I20260811 19:15:20.944540 29436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:20.943392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35949 (local address 127.3.61.129:33133)
0811 19:15:20.944095 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:20.944310 (+ 215us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":362,"threads_started":1}
I20260811 19:15:20.983084 29437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:20.982069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35949 (local address 127.3.61.131:43017)
0811 19:15:20.982754 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:20.982863 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":396,"threads_started":1}
I20260811 19:15:21.465855 29438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:21.464893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35949 (local address 127.3.61.130:37119)
0811 19:15:21.465460 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:21.465575 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":294,"threads_started":1}
I20260811 19:15:21.947070 29443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:21.945886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35949 (local address 127.3.61.129:43183)
0811 19:15:21.946698 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:21.946839 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":377,"threads_started":1}
I20260811 19:15:21.985327 29444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:21.984394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35949 (local address 127.3.61.131:35859)
0811 19:15:21.984989 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:21.985104 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":292,"threads_started":1}
I20260811 19:15:22.468294 29447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:22.467329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35949 (local address 127.3.61.130:60609)
0811 19:15:22.467976 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:22.468094 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":342,"threads_started":1}
I20260811 19:15:22.949898 29449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:22.948506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35949 (local address 127.3.61.129:49925)
0811 19:15:22.949280 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:22.949593 (+ 313us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":624,"thread_start_us":407,"threads_started":1}
I20260811 19:15:22.987325 29450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:22.986494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35949 (local address 127.3.61.131:43157)
0811 19:15:22.987035 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:22.987134 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":271,"threads_started":1}
I20260811 19:15:23.470605 29452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:23.469727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35949 (local address 127.3.61.130:43493)
0811 19:15:23.470288 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:23.470395 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":278,"threads_started":1}
I20260811 19:15:23.951961 29454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:23.951139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35949 (local address 127.3.61.129:39423)
0811 19:15:23.951634 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:23.951736 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":249,"threads_started":1}
I20260811 19:15:23.989409 29455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:23.988511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35949 (local address 127.3.61.131:41253)
0811 19:15:23.989071 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:23.989192 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":274,"threads_started":1}
I20260811 19:15:24.472548 29456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:24.471713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35949 (local address 127.3.61.130:45615)
0811 19:15:24.472222 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:24.472344 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":263,"threads_started":1}
I20260811 19:15:24.953902 29457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:24.953118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35949 (local address 127.3.61.129:55617)
0811 19:15:24.953564 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:24.953702 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":227,"threads_started":1}
I20260811 19:15:24.991282 29462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:24.990518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35949 (local address 127.3.61.131:53375)
0811 19:15:24.991007 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:24.991095 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":248,"threads_started":1}
I20260811 19:15:25.475066 29463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:25.474040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35949 (local address 127.3.61.130:41503)
0811 19:15:25.474670 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:25.474779 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":325,"threads_started":1}
I20260811 19:15:25.956148 29464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:25.955198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35949 (local address 127.3.61.129:37067)
0811 19:15:25.955848 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:25.955968 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":374,"threads_started":1}
I20260811 19:15:25.962608 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 28732
I20260811 19:15:25.992527 29465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:25.964141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53602 (local address 127.3.61.129:43955)
0811 19:15:25.964959 (+ 818us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:25.964975 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:25.965027 (+ 52us) server_negotiation.cc:407] Connection header received
0811 19:15:25.966908 (+ 1881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:25.966920 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:25.967528 (+ 608us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:25.967889 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:15:25.969759 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:25.971166 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:25.973261 (+ 2095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:25.974680 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:25.974958 (+ 278us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:25.983123 (+ 8165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:15:25.983134 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:15:25.983156 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:15:25.983211 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:15:25.986889 (+ 3678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:25.987909 (+ 1020us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:25.987919 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:25.987937 (+ 18us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:25.988176 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:25.988685 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:25.988690 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:25.988693 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:25.990122 (+ 1429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:15:25.990604 (+ 482us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:25.991104 (+ 500us) server_negotiation.cc:299] Negotiation successful
0811 19:15:25.991834 (+ 730us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":624,"thread_start_us":447,"threads_started":1}
I20260811 19:15:25.994059 29467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:25.992886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35949 (local address 127.3.61.131:51953)
0811 19:15:25.993577 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:25.993784 (+ 207us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:35949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":319,"threads_started":1}
I20260811 19:15:25.994719 28975 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:15:26.084933 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 28732
I20260811 19:15:26.112255 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 28987
I20260811 19:15:26.138486 29470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:26.113913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54076 (local address 127.3.61.130:38049)
0811 19:15:26.114723 (+ 810us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:26.114743 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:26.114798 (+ 55us) server_negotiation.cc:407] Connection header received
0811 19:15:26.116836 (+ 2038us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:26.116853 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:26.117486 (+ 633us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:26.117990 (+ 504us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:15:26.119876 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:26.121915 (+ 2039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:26.123745 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:26.124729 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:26.124952 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:26.128764 (+ 3812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:15:26.128776 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:15:26.128799 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:15:26.128861 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:15:26.132733 (+ 3872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:26.134228 (+ 1495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:26.134242 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:26.134251 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:26.134624 (+ 373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:26.135347 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:26.135352 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:26.135355 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:26.136399 (+ 1044us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:15:26.136754 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:26.137402 (+ 648us) server_negotiation.cc:299] Negotiation successful
0811 19:15:26.138099 (+ 697us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":590,"thread_start_us":419,"threads_started":1}
I20260811 19:15:26.140898 29184 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:15:26.240813 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 28987
I20260811 19:15:26.270310 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 29196
I20260811 19:15:26.292351 29474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:26.271666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34508 (local address 127.3.61.131:42375)
0811 19:15:26.272316 (+ 650us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:26.272331 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:26.272381 (+ 50us) server_negotiation.cc:407] Connection header received
0811 19:15:26.274147 (+ 1766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:26.274159 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:26.274766 (+ 607us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:26.275155 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:15:26.276818 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:26.278477 (+ 1659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:26.279898 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:26.280622 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:26.280831 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:26.284228 (+ 3397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:15:26.284241 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:15:26.284269 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:15:26.284348 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:15:26.288294 (+ 3946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:26.289225 (+ 931us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:26.289233 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:26.289242 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:26.289509 (+ 267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:26.290129 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:26.290135 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:26.290139 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:26.290808 (+ 669us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:15:26.291137 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:26.291633 (+ 496us) server_negotiation.cc:299] Negotiation successful
0811 19:15:26.292016 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":313,"threads_started":1}
I20260811 19:15:26.294190 29380 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:15:26.386097 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 29196
2026-08-11T19:15:26Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (25749 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:15:26 dist-test-slave-t7g5 krb5kdc[29479](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:15:26 dist-test-slave-t7g5 krb5kdc[29479](info): set up 2 sockets
Aug 11 19:15:26 dist-test-slave-t7g5 krb5kdc[29479](info): commencing operation
krb5kdc: starting...
W20260811 19:15:27.758890 3318 mini_kdc.cc:121] Time spent starting KDC: real 1.318s user 0.007s sys 0.055s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Aug 11 19:15:27 dist-test-slave-t7g5 krb5kdc[29479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475727, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:15:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:15:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.61.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab.
Entry for principal kudu/127.3.61.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab.
I20260811 19:15:28.133693 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:43283
--webserver_interface=127.3.61.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:35451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.61.190:43283
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:15:28.563066 29501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:15:28.563421 29501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:15:28.563560 29501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:15:28.573132 29501 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 19:15:28.573278 29501 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 19:15:28.573318 29501 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 19:15:28.573374 29501 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:15:28.573462 29501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:15:28.573557 29501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:15:28.573613 29501 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:15:28.573695 29501 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:15:28.585424 29501 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:35451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.3.61.190:43283
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:43283
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.61.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:15:28.586938 29501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:15:28.588456 29501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:15:28.604233 29508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:15:28.604167 29515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:15:28.604137 29509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:15:28.605844 29501 server_base.cc:1034] running on GCE node
I20260811 19:15:28.607038 29501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:15:28.608886 29501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:15:28.610256 29501 hybrid_clock.cc:648] HybridClock initialized: now 1786475728610155 us; error 56 us; skew 500 ppm
I20260811 19:15:28.610839 29501 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:15:28 dist-test-slave-t7g5 krb5kdc[29479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475728, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:15:28.615623 29501 init.cc:377] Logged in from keytab as kudu/127.3.61.190@KRBTEST.COM (short username kudu)
I20260811 19:15:28.618187 29501 webserver.cc:460] Webserver started at http://127.3.61.190:42311/ using document root <none> and password file <none>
I20260811 19:15:28.619324 29501 fs_manager.cc:359] Metadata directory not provided
I20260811 19:15:28.619503 29501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:15:28.619875 29501 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:15:28.623765 29501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/master-0/data/instance:
uuid: "41c09bf6448a4760973e8a7897b4ccd0"
format_stamp: "Formatted at 2026-08-11 19:15:28 on dist-test-slave-t7g5"
server_key: "3d241892a73f429e804e9ec3331fb46e"
server_key_iv: "a47e4b8a88ff5ed8f5c0e2cf7e78072e"
server_key_version: "encryptionkey@0"
I20260811 19:15:28.624795 29501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/master-0/wal/instance:
uuid: "41c09bf6448a4760973e8a7897b4ccd0"
format_stamp: "Formatted at 2026-08-11 19:15:28 on dist-test-slave-t7g5"
server_key: "3d241892a73f429e804e9ec3331fb46e"
server_key_iv: "a47e4b8a88ff5ed8f5c0e2cf7e78072e"
server_key_version: "encryptionkey@0"
I20260811 19:15:28.631635 29501 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260811 19:15:28.637205 29525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:15:28.639251 29501 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260811 19:15:28.639523 29501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/master-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/master-0/wal
uuid: "41c09bf6448a4760973e8a7897b4ccd0"
format_stamp: "Formatted at 2026-08-11 19:15:28 on dist-test-slave-t7g5"
server_key: "3d241892a73f429e804e9ec3331fb46e"
server_key_iv: "a47e4b8a88ff5ed8f5c0e2cf7e78072e"
server_key_version: "encryptionkey@0"
I20260811 19:15:28.639813 29501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:15:28.656033 29501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:15:28.661468 29501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:15:28.661839 29501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:15:28.685082 29501 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.190:43283
I20260811 19:15:28.685099 29604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.190:43283 every 8 connection(s)
I20260811 19:15:28.686969 29501 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
I20260811 19:15:28.692337 29605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:15:28.694478 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 29501
I20260811 19:15:28.694761 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/master-0/wal/instance
I20260811 19:15:28.696225 3318 external_mini_cluster.cc:1428] Setting key 170e32b88d1568b4aa64b4e919359e44
I20260811 19:15:28.709163 29605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0: Bootstrap starting.
I20260811 19:15:28.713186 29605 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0: Neither blocks nor log segments found. Creating new log.
Aug 11 19:15:28 dist-test-slave-t7g5 krb5kdc[29479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475727, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:15:28.714689 29605 log.cc:824] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0: Log is configured to *not* fsync() on all Append() calls
I20260811 19:15:28.720021 29605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0: No bootstrap required, opened a new log
I20260811 19:15:28.728168 29608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:28.699667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53920 (local address 127.3.61.190:43283)
0811 19:15:28.701047 (+ 1380us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:28.701065 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:28.701132 (+ 67us) server_negotiation.cc:407] Connection header received
0811 19:15:28.703370 (+ 2238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:28.703411 (+ 41us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:28.704012 (+ 601us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:28.704486 (+ 474us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:15:28.706656 (+ 2170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:28.708210 (+ 1554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:28.709891 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:28.710690 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:28.710917 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:28.716520 (+ 5603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:15:28.716539 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:15:28.716577 (+ 38us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:15:28.716643 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:15:28.721756 (+ 5113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:28.723000 (+ 1244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:28.723012 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:28.723024 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:28.723327 (+ 303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:28.723918 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:28.723923 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:28.723930 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:28.724887 (+ 957us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:15:28.725269 (+ 382us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:28.725841 (+ 572us) server_negotiation.cc:299] Negotiation successful
0811 19:15:28.726466 (+ 625us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":991,"thread_start_us":603,"threads_started":1}
I20260811 19:15:28.731449 29605 raft_consensus.cc:348] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41c09bf6448a4760973e8a7897b4ccd0" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 43283 } }
I20260811 19:15:28.732008 29605 raft_consensus.cc:374] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:15:28.732136 29605 raft_consensus.cc:729] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41c09bf6448a4760973e8a7897b4ccd0, State: Initialized, Role: FOLLOWER
I20260811 19:15:28.732828 29605 consensus_queue.cc:260] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41c09bf6448a4760973e8a7897b4ccd0" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 43283 } }
I20260811 19:15:28.733117 29605 raft_consensus.cc:388] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:15:28.733244 29605 raft_consensus.cc:482] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:15:28.733445 29605 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:15:28.736173 29605 raft_consensus.cc:504] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41c09bf6448a4760973e8a7897b4ccd0" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 43283 } }
I20260811 19:15:28.736888 29605 leader_election.cc:302] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 41c09bf6448a4760973e8a7897b4ccd0; no voters:
I20260811 19:15:28.737879 29605 leader_election.cc:288] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:15:28.737941 29616 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:15:28.738782 29616 raft_consensus.cc:686] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0 [term 1 LEADER]: Becoming Leader. State: Replica: 41c09bf6448a4760973e8a7897b4ccd0, State: Running, Role: LEADER
I20260811 19:15:28.739529 29616 consensus_queue.cc:237] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41c09bf6448a4760973e8a7897b4ccd0" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 43283 } }
I20260811 19:15:28.742189 29605 sys_catalog.cc:564] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:15:28.742980 29617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 41c09bf6448a4760973e8a7897b4ccd0. Latest consensus state: current_term: 1 leader_uuid: "41c09bf6448a4760973e8a7897b4ccd0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41c09bf6448a4760973e8a7897b4ccd0" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 43283 } } }
I20260811 19:15:28.743733 29617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0 [sys.catalog]: This master's current role is: LEADER
I20260811 19:15:28.742980 29618 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "41c09bf6448a4760973e8a7897b4ccd0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41c09bf6448a4760973e8a7897b4ccd0" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 43283 } } }
I20260811 19:15:28.744689 29618 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0 [sys.catalog]: This master's current role is: LEADER
I20260811 19:15:28.746038 29621 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:15:28.753643 29621 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:15:28.766688 29621 catalog_manager.cc:1347] Generated new cluster ID: 3f91653b5f7b4343ad108648ffeec1e3
I20260811 19:15:28.766880 29621 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:15:28.796908 29621 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:15:28.799283 29621 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:15:28.812371 29621 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0: Generated new TSK 0
I20260811 19:15:28.813738 29621 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.61.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab.
Entry for principal kudu/127.3.61.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab.
I20260811 19:15:28.922410 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.129:0
--local_ip_for_outbound_sockets=127.3.61.129
--webserver_interface=127.3.61.129
--webserver_port=0
--tserver_master_addrs=127.3.61.190:43283
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab
--principal=kudu/127.3.61.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:35451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:15:29.352890 29652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:15:29.353330 29652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:15:29.353480 29652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:15:29.363025 29652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:15:29.363204 29652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:15:29.363344 29652 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.129
I20260811 19:15:29.375222 29652 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:35451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab
--principal=kudu/127.3.61.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.61.129
--webserver_port=0
--tserver_master_addrs=127.3.61.190:43283
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.129
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:15:29.376741 29652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:15:29.378283 29652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:15:29.394436 29669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:15:29.394618 29665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:15:29.394467 29663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:15:29.395627 29652 server_base.cc:1034] running on GCE node
I20260811 19:15:29.396672 29652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:15:29.398090 29652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:15:29.399438 29652 hybrid_clock.cc:648] HybridClock initialized: now 1786475729399348 us; error 96 us; skew 500 ppm
I20260811 19:15:29.400035 29652 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:15:29 dist-test-slave-t7g5 krb5kdc[29479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475729, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:15:29.404251 29652 init.cc:377] Logged in from keytab as kudu/127.3.61.129@KRBTEST.COM (short username kudu)
I20260811 19:15:29.406239 29652 webserver.cc:460] Webserver started at http://127.3.61.129:44995/ using document root <none> and password file <none>
I20260811 19:15:29.407050 29652 fs_manager.cc:359] Metadata directory not provided
I20260811 19:15:29.407194 29652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:15:29.407518 29652 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:15:29.411089 29652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-0/data/instance:
uuid: "4b8d3747e7a240acb769d8b40c9fff76"
format_stamp: "Formatted at 2026-08-11 19:15:29 on dist-test-slave-t7g5"
server_key: "e7f851f8670b06e7bf3d0cd48f0abe3a"
server_key_iv: "f9cbae46258a0c9ac2c14426e0da52d4"
server_key_version: "encryptionkey@0"
I20260811 19:15:29.412052 29652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-0/wal/instance:
uuid: "4b8d3747e7a240acb769d8b40c9fff76"
format_stamp: "Formatted at 2026-08-11 19:15:29 on dist-test-slave-t7g5"
server_key: "e7f851f8670b06e7bf3d0cd48f0abe3a"
server_key_iv: "f9cbae46258a0c9ac2c14426e0da52d4"
server_key_version: "encryptionkey@0"
I20260811 19:15:29.419975 29652 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.001s sys 0.007s
I20260811 19:15:29.425752 29679 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:15:29.428049 29652 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.002s
I20260811 19:15:29.428323 29652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-0/wal
uuid: "4b8d3747e7a240acb769d8b40c9fff76"
format_stamp: "Formatted at 2026-08-11 19:15:29 on dist-test-slave-t7g5"
server_key: "e7f851f8670b06e7bf3d0cd48f0abe3a"
server_key_iv: "f9cbae46258a0c9ac2c14426e0da52d4"
server_key_version: "encryptionkey@0"
I20260811 19:15:29.428596 29652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:15:29.448777 29652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:15:29.456046 29652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:15:29.456647 29652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:15:29.458295 29652 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:15:29.460327 29652 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:15:29.460428 29652 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:15:29.460551 29652 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:15:29.460616 29652 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:15:29.502331 29652 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.129:41967
I20260811 19:15:29.502384 29874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.129:41967 every 8 connection(s)
I20260811 19:15:29.503896 29652 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
I20260811 19:15:29.507709 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 29652
I20260811 19:15:29.508028 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-0/wal/instance
I20260811 19:15:29.508986 3318 external_mini_cluster.cc:1428] Setting key cdd27bd24d212ccd951726fea5209410
Aug 11 19:15:29 dist-test-slave-t7g5 krb5kdc[29479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475729, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.129@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:15:29.534225 29877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:29.508018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:36029 (local address 127.3.61.190:43283)
0811 19:15:29.508655 (+ 637us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:29.508660 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:29.510783 (+ 2123us) server_negotiation.cc:407] Connection header received
0811 19:15:29.513367 (+ 2584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:29.513374 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:29.513493 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:29.513593 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:15:29.515693 (+ 2100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:29.516475 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:29.518818 (+ 2343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:29.519278 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:29.519427 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:29.525465 (+ 6038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:15:29.525474 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:15:29.525478 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:15:29.525534 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:15:29.528950 (+ 3416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:29.530503 (+ 1553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:29.530510 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:29.530514 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:29.530712 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:29.531607 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:29.531614 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:29.531617 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:29.532041 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:15:29.532239 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:29.533462 (+ 1223us) server_negotiation.cc:299] Negotiation successful
0811 19:15:29.533819 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":515,"thread_start_us":379,"threads_started":1}
I20260811 19:15:29.535715 29878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:29.508866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43283 (local address 127.3.61.129:36029)
0811 19:15:29.510579 (+ 1713us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:29.510648 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0811 19:15:29.512990 (+ 2342us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:15:29.513975 (+ 985us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:15:29.513985 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:15:29.514660 (+ 675us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:15:29.515409 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:29.515434 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:29.516693 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:29.516698 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:29.518579 (+ 1881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:29.518603 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:29.519595 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:29.519603 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:29.519887 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:29.520713 (+ 826us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:15:29.520740 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:15:29.524903 (+ 4163us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:15:29.529279 (+ 4376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:15:29.529292 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:15:29.529313 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:15:29.530284 (+ 971us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:15:29.530970 (+ 686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:15:29.530977 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:15:29.530980 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:15:29.531398 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:15:29.532327 (+ 929us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:15:29.532338 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:15:29.532827 (+ 489us) client_negotiation.cc:769] Sending connection context
0811 19:15:29.533214 (+ 387us) client_negotiation.cc:240] Negotiation successful
0811 19:15:29.534058 (+ 844us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1211,"thread_start_us":574,"threads_started":1}
I20260811 19:15:29.538079 29875 heartbeater.cc:347] Connected to a master server at 127.3.61.190:43283
I20260811 19:15:29.538450 29875 heartbeater.cc:464] Registering TS with master...
I20260811 19:15:29.539333 29875 heartbeater.cc:511] Master 127.3.61.190:43283 requested a full tablet report, sending...
I20260811 19:15:29.542686 29551 ts_manager.cc:194] Registered new tserver with Master: 4b8d3747e7a240acb769d8b40c9fff76 (127.3.61.129:41967)
I20260811 19:15:29.545336 29551 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.129@KRBTEST.COM'} at 127.3.61.129:36029
WARNING: no policy specified for kudu/127.3.61.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab.
Entry for principal kudu/127.3.61.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab.
I20260811 19:15:29.602504 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.130:0
--local_ip_for_outbound_sockets=127.3.61.130
--webserver_interface=127.3.61.130
--webserver_port=0
--tserver_master_addrs=127.3.61.190:43283
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab
--principal=kudu/127.3.61.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:35451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:15:30.043721 29882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:15:30.044081 29882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:15:30.044193 29882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:15:30.052240 29882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:15:30.052395 29882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:15:30.052541 29882 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.130
I20260811 19:15:30.064939 29882 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:35451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab
--principal=kudu/127.3.61.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.61.130
--webserver_port=0
--tserver_master_addrs=127.3.61.190:43283
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.130
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:15:30.066852 29882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:15:30.068357 29882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:15:30.084757 29889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:15:30.084712 29891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:15:30.084687 29894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:15:30.086112 29882 server_base.cc:1034] running on GCE node
I20260811 19:15:30.087174 29882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:15:30.088354 29882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:15:30.089728 29882 hybrid_clock.cc:648] HybridClock initialized: now 1786475730089584 us; error 67 us; skew 500 ppm
I20260811 19:15:30.090312 29882 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:15:30 dist-test-slave-t7g5 krb5kdc[29479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475730, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:15:30.095290 29882 init.cc:377] Logged in from keytab as kudu/127.3.61.130@KRBTEST.COM (short username kudu)
I20260811 19:15:30.097432 29882 webserver.cc:460] Webserver started at http://127.3.61.130:43429/ using document root <none> and password file <none>
I20260811 19:15:30.098407 29882 fs_manager.cc:359] Metadata directory not provided
I20260811 19:15:30.098575 29882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:15:30.098968 29882 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:15:30.102882 29882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-1/data/instance:
uuid: "4a86881f7b5949c6b4c66b5501d511a6"
format_stamp: "Formatted at 2026-08-11 19:15:30 on dist-test-slave-t7g5"
server_key: "280d131b916c7795b0661227a79df126"
server_key_iv: "8cd44d4237f74d5e912f97328baed288"
server_key_version: "encryptionkey@0"
I20260811 19:15:30.104223 29882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-1/wal/instance:
uuid: "4a86881f7b5949c6b4c66b5501d511a6"
format_stamp: "Formatted at 2026-08-11 19:15:30 on dist-test-slave-t7g5"
server_key: "280d131b916c7795b0661227a79df126"
server_key_iv: "8cd44d4237f74d5e912f97328baed288"
server_key_version: "encryptionkey@0"
I20260811 19:15:30.113063 29882 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.005s sys 0.004s
I20260811 19:15:30.118076 29907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:15:30.120143 29882 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 19:15:30.120417 29882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-1/wal
uuid: "4a86881f7b5949c6b4c66b5501d511a6"
format_stamp: "Formatted at 2026-08-11 19:15:30 on dist-test-slave-t7g5"
server_key: "280d131b916c7795b0661227a79df126"
server_key_iv: "8cd44d4237f74d5e912f97328baed288"
server_key_version: "encryptionkey@0"
I20260811 19:15:30.120694 29882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:15:30.144248 29882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:15:30.149408 29882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:15:30.149781 29882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:15:30.150892 29882 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:15:30.152402 29882 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:15:30.152485 29882 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:15:30.152585 29882 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:15:30.152642 29882 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:15:30.190712 29882 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.130:41047
I20260811 19:15:30.190755 30087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.130:41047 every 8 connection(s)
I20260811 19:15:30.192684 29882 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
I20260811 19:15:30.194456 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 29882
I20260811 19:15:30.194730 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-1/wal/instance
I20260811 19:15:30.195451 3318 external_mini_cluster.cc:1428] Setting key 02273931bb465dbf9a4c380d8db7db0c
Aug 11 19:15:30 dist-test-slave-t7g5 krb5kdc[29479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475730, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.130@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:15:30.226538 30091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:30.197809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:41967 (local address 127.3.61.190:43283)
0811 19:15:30.198645 (+ 836us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:30.198654 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:30.201787 (+ 3133us) server_negotiation.cc:407] Connection header received
0811 19:15:30.203818 (+ 2031us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:30.203825 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:30.203948 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:30.204112 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:15:30.207090 (+ 2978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:30.207949 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:30.210696 (+ 2747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:30.211215 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:30.211350 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:30.216947 (+ 5597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:15:30.216957 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:15:30.216962 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:15:30.217025 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:15:30.220782 (+ 3757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:30.222463 (+ 1681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:30.222470 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:30.222473 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:30.222727 (+ 254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:30.223485 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:30.223499 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:30.223501 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:30.223908 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:15:30.224200 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:30.225644 (+ 1444us) server_negotiation.cc:299] Negotiation successful
0811 19:15:30.226095 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":473,"threads_started":1}
I20260811 19:15:30.228355 30093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:30.198492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43283 (local address 127.3.61.130:41967)
0811 19:15:30.199881 (+ 1389us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:30.199964 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0811 19:15:30.203212 (+ 3248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:15:30.204546 (+ 1334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:15:30.204562 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:15:30.205728 (+ 1166us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:15:30.206733 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:30.206771 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:30.208188 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:30.208194 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:30.210316 (+ 2122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:30.210355 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:30.211533 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:30.211541 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:30.211884 (+ 343us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:30.212810 (+ 926us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:15:30.212845 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:15:30.216427 (+ 3582us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:15:30.221095 (+ 4668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:15:30.221109 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:15:30.221139 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:15:30.222128 (+ 989us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:15:30.222903 (+ 775us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:15:30.222908 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:15:30.222911 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:15:30.223270 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:15:30.224101 (+ 831us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:15:30.224112 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:15:30.224815 (+ 703us) client_negotiation.cc:769] Sending connection context
0811 19:15:30.225280 (+ 465us) client_negotiation.cc:240] Negotiation successful
0811 19:15:30.226673 (+ 1393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":917,"thread_start_us":541,"threads_started":1}
I20260811 19:15:30.230664 30088 heartbeater.cc:347] Connected to a master server at 127.3.61.190:43283
I20260811 19:15:30.231113 30088 heartbeater.cc:464] Registering TS with master...
I20260811 19:15:30.232036 30088 heartbeater.cc:511] Master 127.3.61.190:43283 requested a full tablet report, sending...
I20260811 19:15:30.234582 29551 ts_manager.cc:194] Registered new tserver with Master: 4a86881f7b5949c6b4c66b5501d511a6 (127.3.61.130:41047)
I20260811 19:15:30.235659 29551 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.130@KRBTEST.COM'} at 127.3.61.130:41967
WARNING: no policy specified for kudu/127.3.61.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab.
Entry for principal kudu/127.3.61.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab.
I20260811 19:15:30.289351 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.131:0
--local_ip_for_outbound_sockets=127.3.61.131
--webserver_interface=127.3.61.131
--webserver_port=0
--tserver_master_addrs=127.3.61.190:43283
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab
--principal=kudu/127.3.61.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:35451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:15:30.549911 29875 heartbeater.cc:503] Master 127.3.61.190:43283 was elected leader, sending a full tablet report...
W20260811 19:15:30.704236 30097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:15:30.704596 30097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:15:30.704699 30097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:15:30.713960 30097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:15:30.714129 30097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:15:30.714258 30097 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.131
I20260811 19:15:30.725963 30097 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:35451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab
--principal=kudu/127.3.61.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.61.131
--webserver_port=0
--tserver_master_addrs=127.3.61.190:43283
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.131
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:15:30.727495 30097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:15:30.728930 30097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:15:30.747023 30108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:15:30.747176 30110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:15:30.747063 30113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:15:30.748385 30097 server_base.cc:1034] running on GCE node
I20260811 19:15:30.749320 30097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:15:30.750638 30097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:15:30.751919 30097 hybrid_clock.cc:648] HybridClock initialized: now 1786475730751857 us; error 50 us; skew 500 ppm
I20260811 19:15:30.752319 30097 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:15:30 dist-test-slave-t7g5 krb5kdc[29479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475730, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:15:30.755987 30097 init.cc:377] Logged in from keytab as kudu/127.3.61.131@KRBTEST.COM (short username kudu)
I20260811 19:15:30.757946 30097 webserver.cc:460] Webserver started at http://127.3.61.131:36513/ using document root <none> and password file <none>
I20260811 19:15:30.758702 30097 fs_manager.cc:359] Metadata directory not provided
I20260811 19:15:30.758817 30097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:15:30.759126 30097 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:15:30.762663 30097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-2/data/instance:
uuid: "0b1650603d4846b0ab3a75db70545cd9"
format_stamp: "Formatted at 2026-08-11 19:15:30 on dist-test-slave-t7g5"
server_key: "57563b6588e292f0c6dac01ca95bbbae"
server_key_iv: "9303a5f0bde5aa3283ddcc2dad2497a8"
server_key_version: "encryptionkey@0"
I20260811 19:15:30.763867 30097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-2/wal/instance:
uuid: "0b1650603d4846b0ab3a75db70545cd9"
format_stamp: "Formatted at 2026-08-11 19:15:30 on dist-test-slave-t7g5"
server_key: "57563b6588e292f0c6dac01ca95bbbae"
server_key_iv: "9303a5f0bde5aa3283ddcc2dad2497a8"
server_key_version: "encryptionkey@0"
I20260811 19:15:30.772490 30097 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.006s sys 0.003s
I20260811 19:15:30.777756 30121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:15:30.779635 30097 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260811 19:15:30.779863 30097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-2/wal
uuid: "0b1650603d4846b0ab3a75db70545cd9"
format_stamp: "Formatted at 2026-08-11 19:15:30 on dist-test-slave-t7g5"
server_key: "57563b6588e292f0c6dac01ca95bbbae"
server_key_iv: "9303a5f0bde5aa3283ddcc2dad2497a8"
server_key_version: "encryptionkey@0"
I20260811 19:15:30.780148 30097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:15:30.799041 30097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:15:30.804776 30097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:15:30.805236 30097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:15:30.806777 30097 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:15:30.808772 30097 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:15:30.808863 30097 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:15:30.808984 30097 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:15:30.809051 30097 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:15:30.850981 30304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.131:37193 every 8 connection(s)
I20260811 19:15:30.851049 30097 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.131:37193
I20260811 19:15:30.852705 30097 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
I20260811 19:15:30.860577 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 30097
I20260811 19:15:30.860776 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-2/wal/instance
I20260811 19:15:30.861310 3318 external_mini_cluster.cc:1428] Setting key 7d7c114fa2c8b8daecf0ea3683719184
Aug 11 19:15:30 dist-test-slave-t7g5 krb5kdc[29479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475730, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.131@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:15:30.881108 30312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:30.856486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:38803 (local address 127.3.61.190:43283)
0811 19:15:30.856995 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:30.857000 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:30.858584 (+ 1584us) server_negotiation.cc:407] Connection header received
0811 19:15:30.861196 (+ 2612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:30.861222 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:30.861387 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:30.861581 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:15:30.863915 (+ 2334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:30.865160 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:30.867078 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:30.867541 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:30.867668 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:30.872799 (+ 5131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:15:30.872809 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:15:30.872813 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:15:30.872877 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:15:30.876532 (+ 3655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:30.877960 (+ 1428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:30.877967 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:30.877969 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:30.878182 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:30.878840 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:30.878845 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:30.878848 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:30.879308 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:15:30.879477 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:30.880431 (+ 954us) server_negotiation.cc:299] Negotiation successful
0811 19:15:30.880738 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":259,"threads_started":1}
I20260811 19:15:30.882076 30315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:30.857185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43283 (local address 127.3.61.131:38803)
0811 19:15:30.858337 (+ 1152us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:30.858398 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 19:15:30.860588 (+ 2190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:15:30.862039 (+ 1451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:15:30.862052 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:15:30.862852 (+ 800us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:15:30.863644 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:30.863677 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:30.865379 (+ 1702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:30.865384 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:30.866858 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:30.866879 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:30.868020 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:30.868029 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:30.868391 (+ 362us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:30.869467 (+ 1076us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:15:30.869508 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:15:30.872276 (+ 2768us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:15:30.876827 (+ 4551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:15:30.876850 (+ 23us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:15:30.876879 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:15:30.877770 (+ 891us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:15:30.878346 (+ 576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:15:30.878350 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:15:30.878353 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:15:30.878666 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:15:30.879451 (+ 785us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:15:30.879459 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:15:30.879899 (+ 440us) client_negotiation.cc:769] Sending connection context
0811 19:15:30.880184 (+ 285us) client_negotiation.cc:240] Negotiation successful
0811 19:15:30.880709 (+ 525us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":781,"thread_start_us":474,"threads_started":1}
I20260811 19:15:30.884034 30305 heartbeater.cc:347] Connected to a master server at 127.3.61.190:43283
I20260811 19:15:30.884457 30305 heartbeater.cc:464] Registering TS with master...
I20260811 19:15:30.885463 30305 heartbeater.cc:511] Master 127.3.61.190:43283 requested a full tablet report, sending...
I20260811 19:15:30.887506 29551 ts_manager.cc:194] Registered new tserver with Master: 0b1650603d4846b0ab3a75db70545cd9 (127.3.61.131:37193)
I20260811 19:15:30.888432 29551 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.131@KRBTEST.COM'} at 127.3.61.131:38803
I20260811 19:15:30.892702 3318 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 19:15:30.917550 30312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:30.899019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53924 (local address 127.3.61.190:43283)
0811 19:15:30.899364 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:30.899369 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:30.899659 (+ 290us) server_negotiation.cc:407] Connection header received
0811 19:15:30.899969 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:30.899975 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:30.900071 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:30.900172 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:15:30.902350 (+ 2178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:30.903288 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:30.905526 (+ 2238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:30.906128 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:30.906297 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:30.909762 (+ 3465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:15:30.909778 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:15:30.909782 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:15:30.909847 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:15:30.913244 (+ 3397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:30.914755 (+ 1511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:30.914763 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:30.914765 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:30.914980 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:30.915594 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:30.915600 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:30.915602 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:30.916178 (+ 576us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:15:30.916569 (+ 391us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:30.916989 (+ 420us) server_negotiation.cc:299] Negotiation successful
0811 19:15:30.917246 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179}
I20260811 19:15:30.922996 29551 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53924:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 19:15:30.932443 29551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:15:30.965054 30335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:30.949585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49066 (local address 127.3.61.130:41047)
0811 19:15:30.953756 (+ 4171us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:30.953768 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:30.954958 (+ 1190us) server_negotiation.cc:407] Connection header received
0811 19:15:30.955185 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:30.955195 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:30.955535 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:30.955772 (+ 237us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:15:30.956883 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:30.958517 (+ 1634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:30.961842 (+ 3325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:30.963617 (+ 1775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:30.963802 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:30.964037 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:30.964510 (+ 473us) server_negotiation.cc:299] Negotiation successful
0811 19:15:30.964709 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2501,"thread_start_us":3401,"threads_started":1}
I20260811 19:15:30.965369 30336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:30.949191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:41967 (local address 127.0.0.1:48750)
0811 19:15:30.951434 (+ 2243us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:30.951488 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 19:15:30.951698 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:15:30.956381 (+ 4683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:15:30.956386 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:15:30.956409 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:15:30.956898 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:30.956911 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:30.959443 (+ 2532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:30.959449 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:30.962008 (+ 2559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:30.962030 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:30.964132 (+ 2102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:30.964138 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:30.964271 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:30.964297 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 19:15:30.964372 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 19:15:30.964712 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1964,"mutex_wait_us":116,"thread_start_us":6588,"threads_started":1}
I20260811 19:15:30.965515 30332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:30.949346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48750 (local address 127.3.61.129:41967)
0811 19:15:30.955736 (+ 6390us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:30.955744 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:30.955772 (+ 28us) server_negotiation.cc:407] Connection header received
0811 19:15:30.955868 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:30.955876 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:30.956138 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:30.957134 (+ 996us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:15:30.957400 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:30.959248 (+ 1848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:30.962262 (+ 3014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:30.963882 (+ 1620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:30.964575 (+ 693us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:30.964831 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:30.964916 (+ 85us) server_negotiation.cc:299] Negotiation successful
0811 19:15:30.965092 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6210,"thread_start_us":6023,"threads_started":1}
I20260811 19:15:30.965832 30331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:30.952004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.131:37193 (local address 127.0.0.1:54068)
0811 19:15:30.956082 (+ 4078us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:30.956098 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 19:15:30.956254 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:15:30.957208 (+ 954us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:15:30.957212 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:15:30.957235 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:15:30.957623 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:30.957637 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:30.959320 (+ 1683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:30.959326 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:30.961851 (+ 2525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:30.961874 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:30.964414 (+ 2540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:30.964420 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:30.964665 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:30.964700 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 19:15:30.964793 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 19:15:30.964940 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3956,"mutex_wait_us":67,"thread_start_us":2469,"threads_started":1}
I20260811 19:15:30.966274 30337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:30.949204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.130:41047 (local address 127.0.0.1:49066)
0811 19:15:30.954821 (+ 5617us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:30.954838 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 19:15:30.954990 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:15:30.956054 (+ 1064us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:15:30.956063 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:15:30.956099 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:15:30.956655 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:30.956671 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:30.958671 (+ 2000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:30.958678 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:30.961489 (+ 2811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:30.961544 (+ 55us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:30.963901 (+ 2357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:30.963909 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:30.964161 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:30.964233 (+ 72us) client_negotiation.cc:769] Sending connection context
0811 19:15:30.964419 (+ 186us) client_negotiation.cc:240] Negotiation successful
0811 19:15:30.965894 (+ 1475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5426,"mutex_wait_us":710,"thread_start_us":1194,"threads_started":1}
I20260811 19:15:30.967067 30338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:30.952685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54068 (local address 127.3.61.131:37193)
0811 19:15:30.955191 (+ 2506us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:30.955200 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:30.956399 (+ 1199us) server_negotiation.cc:407] Connection header received
0811 19:15:30.956518 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:30.956526 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:30.956724 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:30.956926 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:15:30.957808 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:30.959065 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:30.962254 (+ 3189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:30.963848 (+ 1594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:30.964006 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:30.964239 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:30.966495 (+ 2256us) server_negotiation.cc:299] Negotiation successful
0811 19:15:30.966713 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1463,"thread_start_us":1283,"threads_started":1}
I20260811 19:15:30.969972 29762 tablet_service.cc:1467] Processing CreateTablet for tablet b808b216a79747729252ab0245e3ab00 (DEFAULT_TABLE table=test-table [id=1711f80bb324441dba15658c832eaf01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:15:30.970585 30192 tablet_service.cc:1467] Processing CreateTablet for tablet b808b216a79747729252ab0245e3ab00 (DEFAULT_TABLE table=test-table [id=1711f80bb324441dba15658c832eaf01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:15:30.970585 29980 tablet_service.cc:1467] Processing CreateTablet for tablet b808b216a79747729252ab0245e3ab00 (DEFAULT_TABLE table=test-table [id=1711f80bb324441dba15658c832eaf01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:15:30.971953 29762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b808b216a79747729252ab0245e3ab00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:15:30.972213 30192 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b808b216a79747729252ab0245e3ab00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:15:30.972213 29980 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b808b216a79747729252ab0245e3ab00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:15:30.987849 30342 tablet_bootstrap.cc:492] T b808b216a79747729252ab0245e3ab00 P 4b8d3747e7a240acb769d8b40c9fff76: Bootstrap starting.
I20260811 19:15:30.990551 30344 tablet_bootstrap.cc:492] T b808b216a79747729252ab0245e3ab00 P 4a86881f7b5949c6b4c66b5501d511a6: Bootstrap starting.
I20260811 19:15:30.991580 30345 tablet_bootstrap.cc:492] T b808b216a79747729252ab0245e3ab00 P 0b1650603d4846b0ab3a75db70545cd9: Bootstrap starting.
I20260811 19:15:30.994045 30342 tablet_bootstrap.cc:654] T b808b216a79747729252ab0245e3ab00 P 4b8d3747e7a240acb769d8b40c9fff76: Neither blocks nor log segments found. Creating new log.
I20260811 19:15:30.995383 30342 log.cc:824] T b808b216a79747729252ab0245e3ab00 P 4b8d3747e7a240acb769d8b40c9fff76: Log is configured to *not* fsync() on all Append() calls
I20260811 19:15:30.996206 30345 tablet_bootstrap.cc:654] T b808b216a79747729252ab0245e3ab00 P 0b1650603d4846b0ab3a75db70545cd9: Neither blocks nor log segments found. Creating new log.
I20260811 19:15:30.996409 30344 tablet_bootstrap.cc:654] T b808b216a79747729252ab0245e3ab00 P 4a86881f7b5949c6b4c66b5501d511a6: Neither blocks nor log segments found. Creating new log.
I20260811 19:15:30.997784 30344 log.cc:824] T b808b216a79747729252ab0245e3ab00 P 4a86881f7b5949c6b4c66b5501d511a6: Log is configured to *not* fsync() on all Append() calls
I20260811 19:15:30.997777 30345 log.cc:824] T b808b216a79747729252ab0245e3ab00 P 0b1650603d4846b0ab3a75db70545cd9: Log is configured to *not* fsync() on all Append() calls
I20260811 19:15:31.001693 30344 tablet_bootstrap.cc:492] T b808b216a79747729252ab0245e3ab00 P 4a86881f7b5949c6b4c66b5501d511a6: No bootstrap required, opened a new log
I20260811 19:15:31.001789 30345 tablet_bootstrap.cc:492] T b808b216a79747729252ab0245e3ab00 P 0b1650603d4846b0ab3a75db70545cd9: No bootstrap required, opened a new log
I20260811 19:15:31.001780 30342 tablet_bootstrap.cc:492] T b808b216a79747729252ab0245e3ab00 P 4b8d3747e7a240acb769d8b40c9fff76: No bootstrap required, opened a new log
I20260811 19:15:31.002084 30344 ts_tablet_manager.cc:1397] T b808b216a79747729252ab0245e3ab00 P 4a86881f7b5949c6b4c66b5501d511a6: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.002s
I20260811 19:15:31.002103 30342 ts_tablet_manager.cc:1397] T b808b216a79747729252ab0245e3ab00 P 4b8d3747e7a240acb769d8b40c9fff76: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.003s
I20260811 19:15:31.002106 30345 ts_tablet_manager.cc:1397] T b808b216a79747729252ab0245e3ab00 P 0b1650603d4846b0ab3a75db70545cd9: Time spent bootstrapping tablet: real 0.011s user 0.010s sys 0.000s
I20260811 19:15:31.011052 30344 raft_consensus.cc:348] T b808b216a79747729252ab0245e3ab00 P 4a86881f7b5949c6b4c66b5501d511a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b8d3747e7a240acb769d8b40c9fff76" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 41967 } } peers { permanent_uuid: "4a86881f7b5949c6b4c66b5501d511a6" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41047 } } peers { permanent_uuid: "0b1650603d4846b0ab3a75db70545cd9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37193 } }
I20260811 19:15:31.011606 30344 raft_consensus.cc:374] T b808b216a79747729252ab0245e3ab00 P 4a86881f7b5949c6b4c66b5501d511a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:15:31.011734 30344 raft_consensus.cc:729] T b808b216a79747729252ab0245e3ab00 P 4a86881f7b5949c6b4c66b5501d511a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a86881f7b5949c6b4c66b5501d511a6, State: Initialized, Role: FOLLOWER
I20260811 19:15:31.012482 30344 consensus_queue.cc:260] T b808b216a79747729252ab0245e3ab00 P 4a86881f7b5949c6b4c66b5501d511a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b8d3747e7a240acb769d8b40c9fff76" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 41967 } } peers { permanent_uuid: "4a86881f7b5949c6b4c66b5501d511a6" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41047 } } peers { permanent_uuid: "0b1650603d4846b0ab3a75db70545cd9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37193 } }
I20260811 19:15:31.013856 30088 heartbeater.cc:503] Master 127.3.61.190:43283 was elected leader, sending a full tablet report...
I20260811 19:15:31.014117 30345 raft_consensus.cc:348] T b808b216a79747729252ab0245e3ab00 P 0b1650603d4846b0ab3a75db70545cd9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b8d3747e7a240acb769d8b40c9fff76" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 41967 } } peers { permanent_uuid: "4a86881f7b5949c6b4c66b5501d511a6" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41047 } } peers { permanent_uuid: "0b1650603d4846b0ab3a75db70545cd9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37193 } }
I20260811 19:15:31.014117 30342 raft_consensus.cc:348] T b808b216a79747729252ab0245e3ab00 P 4b8d3747e7a240acb769d8b40c9fff76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b8d3747e7a240acb769d8b40c9fff76" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 41967 } } peers { permanent_uuid: "4a86881f7b5949c6b4c66b5501d511a6" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41047 } } peers { permanent_uuid: "0b1650603d4846b0ab3a75db70545cd9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37193 } }
I20260811 19:15:31.014683 30344 ts_tablet_manager.cc:1428] T b808b216a79747729252ab0245e3ab00 P 4a86881f7b5949c6b4c66b5501d511a6: Time spent starting tablet: real 0.012s user 0.011s sys 0.001s
I20260811 19:15:31.014814 30342 raft_consensus.cc:374] T b808b216a79747729252ab0245e3ab00 P 4b8d3747e7a240acb769d8b40c9fff76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:15:31.014814 30345 raft_consensus.cc:374] T b808b216a79747729252ab0245e3ab00 P 0b1650603d4846b0ab3a75db70545cd9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:15:31.014925 30345 raft_consensus.cc:729] T b808b216a79747729252ab0245e3ab00 P 0b1650603d4846b0ab3a75db70545cd9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b1650603d4846b0ab3a75db70545cd9, State: Initialized, Role: FOLLOWER
I20260811 19:15:31.014925 30342 raft_consensus.cc:729] T b808b216a79747729252ab0245e3ab00 P 4b8d3747e7a240acb769d8b40c9fff76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b8d3747e7a240acb769d8b40c9fff76, State: Initialized, Role: FOLLOWER
I20260811 19:15:31.015709 30345 consensus_queue.cc:260] T b808b216a79747729252ab0245e3ab00 P 0b1650603d4846b0ab3a75db70545cd9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b8d3747e7a240acb769d8b40c9fff76" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 41967 } } peers { permanent_uuid: "4a86881f7b5949c6b4c66b5501d511a6" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41047 } } peers { permanent_uuid: "0b1650603d4846b0ab3a75db70545cd9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37193 } }
I20260811 19:15:31.015709 30342 consensus_queue.cc:260] T b808b216a79747729252ab0245e3ab00 P 4b8d3747e7a240acb769d8b40c9fff76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b8d3747e7a240acb769d8b40c9fff76" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 41967 } } peers { permanent_uuid: "4a86881f7b5949c6b4c66b5501d511a6" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41047 } } peers { permanent_uuid: "0b1650603d4846b0ab3a75db70545cd9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37193 } }
I20260811 19:15:31.017030 30305 heartbeater.cc:503] Master 127.3.61.190:43283 was elected leader, sending a full tablet report...
I20260811 19:15:31.019591 30345 ts_tablet_manager.cc:1428] T b808b216a79747729252ab0245e3ab00 P 0b1650603d4846b0ab3a75db70545cd9: Time spent starting tablet: real 0.017s user 0.014s sys 0.000s
I20260811 19:15:31.019591 30342 ts_tablet_manager.cc:1428] T b808b216a79747729252ab0245e3ab00 P 4b8d3747e7a240acb769d8b40c9fff76: Time spent starting tablet: real 0.017s user 0.007s sys 0.008s
I20260811 19:15:31.091485 30353 raft_consensus.cc:482] T b808b216a79747729252ab0245e3ab00 P 4b8d3747e7a240acb769d8b40c9fff76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:15:31.091806 30353 raft_consensus.cc:504] T b808b216a79747729252ab0245e3ab00 P 4b8d3747e7a240acb769d8b40c9fff76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b8d3747e7a240acb769d8b40c9fff76" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 41967 } } peers { permanent_uuid: "4a86881f7b5949c6b4c66b5501d511a6" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41047 } } peers { permanent_uuid: "0b1650603d4846b0ab3a75db70545cd9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37193 } }
I20260811 19:15:31.093616 30353 leader_election.cc:288] T b808b216a79747729252ab0245e3ab00 P 4b8d3747e7a240acb769d8b40c9fff76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4a86881f7b5949c6b4c66b5501d511a6 (127.3.61.130:41047), 0b1650603d4846b0ab3a75db70545cd9 (127.3.61.131:37193)
I20260811 19:15:31.096958 30352 raft_consensus.cc:482] T b808b216a79747729252ab0245e3ab00 P 4a86881f7b5949c6b4c66b5501d511a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:15:31.097348 30352 raft_consensus.cc:504] T b808b216a79747729252ab0245e3ab00 P 4a86881f7b5949c6b4c66b5501d511a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b8d3747e7a240acb769d8b40c9fff76" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 41967 } } peers { permanent_uuid: "4a86881f7b5949c6b4c66b5501d511a6" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41047 } } peers { permanent_uuid: "0b1650603d4846b0ab3a75db70545cd9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37193 } }
I20260811 19:15:31.104187 30338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:31.095451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:57165 (local address 127.3.61.131:37193)
0811 19:15:31.095671 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:31.095677 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:31.096009 (+ 332us) server_negotiation.cc:407] Connection header received
0811 19:15:31.096306 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:31.096312 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:31.096386 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:31.096487 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:15:31.097061 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:31.097942 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:31.101188 (+ 3246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:31.102603 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:31.102775 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:31.102869 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:31.103657 (+ 788us) server_negotiation.cc:299] Negotiation successful
0811 19:15:31.103836 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260811 19:15:31.104786 30308 tablet.cc:2369] T b808b216a79747729252ab0245e3ab00 P 0b1650603d4846b0ab3a75db70545cd9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:15:31.105871 30357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:31.095169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.131:37193 (local address 127.3.61.129:57165)
0811 19:15:31.095881 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:31.095897 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 19:15:31.096144 (+ 247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:15:31.096514 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:15:31.096517 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:15:31.096539 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:15:31.096895 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:31.096914 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:31.098119 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:31.098124 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:31.100726 (+ 2602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:31.100754 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:31.103142 (+ 2388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:31.103150 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:31.103399 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:31.103443 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 19:15:31.105121 (+ 1678us) client_negotiation.cc:240] Negotiation successful
0811 19:15:31.105353 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"mutex_wait_us":8,"thread_start_us":362,"threads_started":1}
I20260811 19:15:31.106426 30335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:31.094279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:42115 (local address 127.3.61.130:41047)
0811 19:15:31.094532 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:31.094541 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:31.095145 (+ 604us) server_negotiation.cc:407] Connection header received
0811 19:15:31.095592 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:31.095598 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:31.095682 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:31.095748 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:15:31.096500 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:31.097629 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:31.100760 (+ 3131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:31.102732 (+ 1972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:31.102925 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:31.103011 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:31.103882 (+ 871us) server_negotiation.cc:299] Negotiation successful
0811 19:15:31.104092 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 19:15:31.107210 30227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b808b216a79747729252ab0245e3ab00" candidate_uuid: "4b8d3747e7a240acb769d8b40c9fff76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b1650603d4846b0ab3a75db70545cd9" is_pre_election: true
I20260811 19:15:31.107594 30356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:31.094165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.130:41047 (local address 127.3.61.129:42115)
0811 19:15:31.095008 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:31.095025 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 19:15:31.095303 (+ 278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:15:31.095929 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:15:31.095933 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:15:31.095962 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:15:31.096336 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:31.096346 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:31.097817 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:31.097823 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:31.100395 (+ 2572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:31.100433 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:31.103384 (+ 2951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:31.103391 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:31.103634 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:31.103672 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 19:15:31.106887 (+ 3215us) client_negotiation.cc:240] Negotiation successful
0811 19:15:31.107099 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":723,"mutex_wait_us":126,"spinlock_wait_cycles":36608,"thread_start_us":473,"threads_started":1}
I20260811 19:15:31.107913 30227 raft_consensus.cc:2454] T b808b216a79747729252ab0245e3ab00 P 0b1650603d4846b0ab3a75db70545cd9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b8d3747e7a240acb769d8b40c9fff76 in term 0.
I20260811 19:15:31.108783 29683 leader_election.cc:302] T b808b216a79747729252ab0245e3ab00 P 4b8d3747e7a240acb769d8b40c9fff76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b1650603d4846b0ab3a75db70545cd9, 4b8d3747e7a240acb769d8b40c9fff76; no voters:
I20260811 19:15:31.108799 30013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b808b216a79747729252ab0245e3ab00" candidate_uuid: "4b8d3747e7a240acb769d8b40c9fff76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a86881f7b5949c6b4c66b5501d511a6" is_pre_election: true
I20260811 19:15:31.109227 30353 raft_consensus.cc:2781] T b808b216a79747729252ab0245e3ab00 P 4b8d3747e7a240acb769d8b40c9fff76 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:15:31.109411 30353 raft_consensus.cc:482] T b808b216a79747729252ab0245e3ab00 P 4b8d3747e7a240acb769d8b40c9fff76 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:15:31.109512 30353 raft_consensus.cc:3037] T b808b216a79747729252ab0245e3ab00 P 4b8d3747e7a240acb769d8b40c9fff76 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:15:31.109401 30013 raft_consensus.cc:2454] T b808b216a79747729252ab0245e3ab00 P 4a86881f7b5949c6b4c66b5501d511a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b8d3747e7a240acb769d8b40c9fff76 in term 0.
I20260811 19:15:31.112001 30352 leader_election.cc:288] T b808b216a79747729252ab0245e3ab00 P 4a86881f7b5949c6b4c66b5501d511a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b8d3747e7a240acb769d8b40c9fff76 (127.3.61.129:41967), 0b1650603d4846b0ab3a75db70545cd9 (127.3.61.131:37193)
I20260811 19:15:31.113106 30353 raft_consensus.cc:504] T b808b216a79747729252ab0245e3ab00 P 4b8d3747e7a240acb769d8b40c9fff76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b8d3747e7a240acb769d8b40c9fff76" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 41967 } } peers { permanent_uuid: "4a86881f7b5949c6b4c66b5501d511a6" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41047 } } peers { permanent_uuid: "0b1650603d4846b0ab3a75db70545cd9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37193 } }
I20260811 19:15:31.114454 30353 leader_election.cc:288] T b808b216a79747729252ab0245e3ab00 P 4b8d3747e7a240acb769d8b40c9fff76 [CANDIDATE]: Term 1 election: Requested vote from peers 4a86881f7b5949c6b4c66b5501d511a6 (127.3.61.130:41047), 0b1650603d4846b0ab3a75db70545cd9 (127.3.61.131:37193)
I20260811 19:15:31.114512 30013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b808b216a79747729252ab0245e3ab00" candidate_uuid: "4b8d3747e7a240acb769d8b40c9fff76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a86881f7b5949c6b4c66b5501d511a6"
I20260811 19:15:31.114737 30013 raft_consensus.cc:3037] T b808b216a79747729252ab0245e3ab00 P 4a86881f7b5949c6b4c66b5501d511a6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:15:31.115612 30227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b808b216a79747729252ab0245e3ab00" candidate_uuid: "4b8d3747e7a240acb769d8b40c9fff76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b1650603d4846b0ab3a75db70545cd9"
I20260811 19:15:31.115864 30227 raft_consensus.cc:3037] T b808b216a79747729252ab0245e3ab00 P 0b1650603d4846b0ab3a75db70545cd9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:15:31.119014 30013 raft_consensus.cc:2454] T b808b216a79747729252ab0245e3ab00 P 4a86881f7b5949c6b4c66b5501d511a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b8d3747e7a240acb769d8b40c9fff76 in term 1.
I20260811 19:15:31.119125 30227 raft_consensus.cc:2454] T b808b216a79747729252ab0245e3ab00 P 0b1650603d4846b0ab3a75db70545cd9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b8d3747e7a240acb769d8b40c9fff76 in term 1.
I20260811 19:15:31.119649 29683 leader_election.cc:302] T b808b216a79747729252ab0245e3ab00 P 4b8d3747e7a240acb769d8b40c9fff76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b1650603d4846b0ab3a75db70545cd9, 4b8d3747e7a240acb769d8b40c9fff76; no voters:
I20260811 19:15:31.120244 30353 raft_consensus.cc:2781] T b808b216a79747729252ab0245e3ab00 P 4b8d3747e7a240acb769d8b40c9fff76 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:15:31.120453 30332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:31.100840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:45517 (local address 127.3.61.129:41967)
0811 19:15:31.101069 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:31.101076 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:31.112547 (+ 11471us) server_negotiation.cc:407] Connection header received
0811 19:15:31.112730 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:31.112735 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:31.112814 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:31.112914 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:15:31.113542 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:31.114671 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:31.118021 (+ 3350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:31.119248 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:31.119392 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:31.119469 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:31.120042 (+ 573us) server_negotiation.cc:299] Negotiation successful
0811 19:15:31.120170 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 19:15:31.120474 30358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:31.100279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:41967 (local address 127.3.61.130:45517)
0811 19:15:31.112425 (+ 12146us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:31.112444 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 19:15:31.112600 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:15:31.112945 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:15:31.112949 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:15:31.112978 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:15:31.113348 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:31.113359 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:31.114819 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:31.114837 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:31.117791 (+ 2954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:31.117805 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:31.119689 (+ 1884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:31.119695 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:31.119859 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:31.119893 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 19:15:31.119981 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 19:15:31.120122 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12003,"thread_start_us":12388,"threads_started":1}
I20260811 19:15:31.121210 30353 raft_consensus.cc:686] T b808b216a79747729252ab0245e3ab00 P 4b8d3747e7a240acb769d8b40c9fff76 [term 1 LEADER]: Becoming Leader. State: Replica: 4b8d3747e7a240acb769d8b40c9fff76, State: Running, Role: LEADER
I20260811 19:15:31.121737 29795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b808b216a79747729252ab0245e3ab00" candidate_uuid: "4a86881f7b5949c6b4c66b5501d511a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b8d3747e7a240acb769d8b40c9fff76" is_pre_election: true
I20260811 19:15:31.122387 30360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:31.100306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.131:37193 (local address 127.3.61.130:51557)
0811 19:15:31.114052 (+ 13746us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:31.114074 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 19:15:31.114285 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:15:31.114819 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:15:31.114824 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:15:31.114850 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:15:31.115837 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:31.115853 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:31.117575 (+ 1722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:31.117581 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:31.119679 (+ 2098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:31.119690 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:31.121534 (+ 1844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:31.121541 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:31.121746 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:31.121781 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 19:15:31.121873 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 19:15:31.122007 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13614,"thread_start_us":13363,"threads_started":1}
I20260811 19:15:31.122248 30353 consensus_queue.cc:237] T b808b216a79747729252ab0245e3ab00 P 4b8d3747e7a240acb769d8b40c9fff76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b8d3747e7a240acb769d8b40c9fff76" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 41967 } } peers { permanent_uuid: "4a86881f7b5949c6b4c66b5501d511a6" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41047 } } peers { permanent_uuid: "0b1650603d4846b0ab3a75db70545cd9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37193 } }
I20260811 19:15:31.123066 30338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:31.100963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:51557 (local address 127.3.61.131:37193)
0811 19:15:31.104960 (+ 3997us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:31.104968 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:31.114173 (+ 9205us) server_negotiation.cc:407] Connection header received
0811 19:15:31.114460 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:31.114466 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:31.114558 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:31.114622 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:15:31.116057 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:31.117361 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:31.119879 (+ 2518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:31.121130 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:31.121310 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:31.121382 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:31.122644 (+ 1262us) server_negotiation.cc:299] Negotiation successful
0811 19:15:31.122792 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3834,"thread_start_us":5610,"threads_started":1}
I20260811 19:15:31.123695 30227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b808b216a79747729252ab0245e3ab00" candidate_uuid: "4a86881f7b5949c6b4c66b5501d511a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b1650603d4846b0ab3a75db70545cd9" is_pre_election: true
I20260811 19:15:31.124024 30227 raft_consensus.cc:2379] T b808b216a79747729252ab0245e3ab00 P 0b1650603d4846b0ab3a75db70545cd9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4a86881f7b5949c6b4c66b5501d511a6 in current term 1: Already voted for candidate 4b8d3747e7a240acb769d8b40c9fff76 in this term.
I20260811 19:15:31.125038 29913 leader_election.cc:302] T b808b216a79747729252ab0245e3ab00 P 4a86881f7b5949c6b4c66b5501d511a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4a86881f7b5949c6b4c66b5501d511a6; no voters: 0b1650603d4846b0ab3a75db70545cd9, 4b8d3747e7a240acb769d8b40c9fff76
I20260811 19:15:31.125854 30352 raft_consensus.cc:2726] T b808b216a79747729252ab0245e3ab00 P 4a86881f7b5949c6b4c66b5501d511a6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 19:15:31.128296 29551 catalog_manager.cc:5495] T b808b216a79747729252ab0245e3ab00 P 4b8d3747e7a240acb769d8b40c9fff76 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b8d3747e7a240acb769d8b40c9fff76 (127.3.61.129). New cstate: current_term: 1 leader_uuid: "4b8d3747e7a240acb769d8b40c9fff76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b8d3747e7a240acb769d8b40c9fff76" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 41967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a86881f7b5949c6b4c66b5501d511a6" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41047 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b1650603d4846b0ab3a75db70545cd9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37193 } health_report { overall_health: UNKNOWN } } }
I20260811 19:15:31.165308 30332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:31.158280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48754 (local address 127.3.61.129:41967)
0811 19:15:31.158459 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:31.158464 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:31.158483 (+ 19us) server_negotiation.cc:407] Connection header received
0811 19:15:31.158588 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:31.158592 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:31.158700 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:31.158849 (+ 149us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:15:31.159599 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:31.160506 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:31.162714 (+ 2208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:31.163219 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:31.163359 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:31.163991 (+ 632us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:15:31.164607 (+ 616us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:15:31.164741 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:31.164930 (+ 189us) server_negotiation.cc:299] Negotiation successful
0811 19:15:31.165021 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 19:15:31.175045 30227 raft_consensus.cc:1261] T b808b216a79747729252ab0245e3ab00 P 0b1650603d4846b0ab3a75db70545cd9 [term 1 FOLLOWER]: Refusing update from remote peer 4b8d3747e7a240acb769d8b40c9fff76: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:15:31.175169 30013 raft_consensus.cc:1261] T b808b216a79747729252ab0245e3ab00 P 4a86881f7b5949c6b4c66b5501d511a6 [term 1 FOLLOWER]: Refusing update from remote peer 4b8d3747e7a240acb769d8b40c9fff76: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:15:31.176474 30363 consensus_queue.cc:1035] T b808b216a79747729252ab0245e3ab00 P 4b8d3747e7a240acb769d8b40c9fff76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b1650603d4846b0ab3a75db70545cd9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:15:31.176962 30353 consensus_queue.cc:1035] T b808b216a79747729252ab0245e3ab00 P 4b8d3747e7a240acb769d8b40c9fff76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a86881f7b5949c6b4c66b5501d511a6" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:15:31.193779 30372 mvcc.cc:205] Tried to move back new op lower bound from 7317404594880188416 to 7317404594681552896. Current Snapshot: MvccSnapshot[applied={T|T < 7317404594880188416}]
I20260811 19:15:31.212167 29551 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53924:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:15:31.212499 29551 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53924:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:15:31.217161 29551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0: Sending DeleteTablet for 3 replicas of tablet b808b216a79747729252ab0245e3ab00
I20260811 19:15:31.218549 29762 tablet_service.cc:1514] Processing DeleteTablet for tablet b808b216a79747729252ab0245e3ab00 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:15:31 UTC) from {username='kudu', principal='kudu/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:48750
I20260811 19:15:31.219156 29980 tablet_service.cc:1514] Processing DeleteTablet for tablet b808b216a79747729252ab0245e3ab00 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:15:31 UTC) from {username='kudu', principal='kudu/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:49066
I20260811 19:15:31.219200 30192 tablet_service.cc:1514] Processing DeleteTablet for tablet b808b216a79747729252ab0245e3ab00 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:15:31 UTC) from {username='kudu', principal='kudu/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:54068
I20260811 19:15:31.220358 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 29652
I20260811 19:15:31.231859 30388 tablet_replica.cc:330] T b808b216a79747729252ab0245e3ab00 P 4a86881f7b5949c6b4c66b5501d511a6: stopping tablet replica
I20260811 19:15:31.232460 30388 raft_consensus.cc:2229] T b808b216a79747729252ab0245e3ab00 P 4a86881f7b5949c6b4c66b5501d511a6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:15:31.233075 30388 raft_consensus.cc:2258] T b808b216a79747729252ab0245e3ab00 P 4a86881f7b5949c6b4c66b5501d511a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:15:31.235776 30388 ts_tablet_manager.cc:1905] T b808b216a79747729252ab0245e3ab00 P 4a86881f7b5949c6b4c66b5501d511a6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:15:31.238322 30387 tablet_replica.cc:330] T b808b216a79747729252ab0245e3ab00 P 4b8d3747e7a240acb769d8b40c9fff76: stopping tablet replica
I20260811 19:15:31.238888 30387 raft_consensus.cc:2229] T b808b216a79747729252ab0245e3ab00 P 4b8d3747e7a240acb769d8b40c9fff76 [term 1 LEADER]: Raft consensus shutting down.
I20260811 19:15:31.239665 30387 raft_consensus.cc:2258] T b808b216a79747729252ab0245e3ab00 P 4b8d3747e7a240acb769d8b40c9fff76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:15:31.239928 30390 tablet_replica.cc:330] T b808b216a79747729252ab0245e3ab00 P 0b1650603d4846b0ab3a75db70545cd9: stopping tablet replica
I20260811 19:15:31.240651 30390 raft_consensus.cc:2229] T b808b216a79747729252ab0245e3ab00 P 0b1650603d4846b0ab3a75db70545cd9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:15:31.241420 30390 raft_consensus.cc:2258] T b808b216a79747729252ab0245e3ab00 P 0b1650603d4846b0ab3a75db70545cd9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:15:31.242530 30387 ts_tablet_manager.cc:1905] T b808b216a79747729252ab0245e3ab00 P 4b8d3747e7a240acb769d8b40c9fff76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:15:31.244222 30390 ts_tablet_manager.cc:1905] T b808b216a79747729252ab0245e3ab00 P 0b1650603d4846b0ab3a75db70545cd9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:15:31.251622 30390 ts_tablet_manager.cc:1918] T b808b216a79747729252ab0245e3ab00 P 0b1650603d4846b0ab3a75db70545cd9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:15:31.251852 30390 log.cc:1196] T b808b216a79747729252ab0245e3ab00 P 0b1650603d4846b0ab3a75db70545cd9: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-2/wal/wals/b808b216a79747729252ab0245e3ab00
I20260811 19:15:31.252446 30390 ts_tablet_manager.cc:1939] T b808b216a79747729252ab0245e3ab00 P 0b1650603d4846b0ab3a75db70545cd9: Deleting consensus metadata
I20260811 19:15:31.253805 30387 ts_tablet_manager.cc:1918] T b808b216a79747729252ab0245e3ab00 P 4b8d3747e7a240acb769d8b40c9fff76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:15:31.254200 30387 log.cc:1196] T b808b216a79747729252ab0245e3ab00 P 4b8d3747e7a240acb769d8b40c9fff76: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-0/wal/wals/b808b216a79747729252ab0245e3ab00
I20260811 19:15:31.254810 30387 ts_tablet_manager.cc:1939] T b808b216a79747729252ab0245e3ab00 P 4b8d3747e7a240acb769d8b40c9fff76: Deleting consensus metadata
I20260811 19:15:31.254814 29528 catalog_manager.cc:4841] TS 0b1650603d4846b0ab3a75db70545cd9 (127.3.61.131:37193): tablet b808b216a79747729252ab0245e3ab00 (table test-table [id=1711f80bb324441dba15658c832eaf01]) successfully deleted
I20260811 19:15:31.255653 30388 ts_tablet_manager.cc:1918] T b808b216a79747729252ab0245e3ab00 P 4a86881f7b5949c6b4c66b5501d511a6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:15:31.256568 30388 log.cc:1196] T b808b216a79747729252ab0245e3ab00 P 4a86881f7b5949c6b4c66b5501d511a6: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-1/wal/wals/b808b216a79747729252ab0245e3ab00
I20260811 19:15:31.256675 29526 catalog_manager.cc:4841] TS 4b8d3747e7a240acb769d8b40c9fff76 (127.3.61.129:41967): tablet b808b216a79747729252ab0245e3ab00 (table test-table [id=1711f80bb324441dba15658c832eaf01]) successfully deleted
I20260811 19:15:31.257102 30388 ts_tablet_manager.cc:1939] T b808b216a79747729252ab0245e3ab00 P 4a86881f7b5949c6b4c66b5501d511a6: Deleting consensus metadata
I20260811 19:15:31.258814 29528 catalog_manager.cc:4841] TS 4a86881f7b5949c6b4c66b5501d511a6 (127.3.61.130:41047): tablet b808b216a79747729252ab0245e3ab00 (table test-table [id=1711f80bb324441dba15658c832eaf01]) successfully deleted
Aug 11 19:15:31 dist-test-slave-t7g5 krb5kdc[29479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475727, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.129@KRBTEST.COM
I20260811 19:15:31.269542 30332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:31.221839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48762 (local address 127.3.61.129:41967)
0811 19:15:31.222055 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:31.222063 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:31.239015 (+ 16952us) server_negotiation.cc:407] Connection header received
0811 19:15:31.239261 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:31.239267 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:31.239446 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:31.239599 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:15:31.242856 (+ 3257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:31.244342 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:31.247589 (+ 3247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:31.248597 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:31.248769 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:31.261541 (+ 12772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:15:31.261561 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:15:31.261579 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:15:31.261625 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:15:31.265259 (+ 3634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:31.266796 (+ 1537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:31.266809 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:31.266812 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:31.267083 (+ 271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:31.267723 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:31.267729 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:31.267732 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:31.268277 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:15:31.268678 (+ 401us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:31.269013 (+ 335us) server_negotiation.cc:299] Negotiation successful
0811 19:15:31.269253 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 19:15:31.270484 29864 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:15:31.377941 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 29652
I20260811 19:15:31.413478 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 29882
Aug 11 19:15:31 dist-test-slave-t7g5 krb5kdc[29479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475727, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.130@KRBTEST.COM
I20260811 19:15:31.433372 30335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:31.414779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49080 (local address 127.3.61.130:41047)
0811 19:15:31.415027 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:31.415032 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:31.415059 (+ 27us) server_negotiation.cc:407] Connection header received
0811 19:15:31.415279 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:31.415285 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:31.415418 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:31.415575 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:15:31.417375 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:31.418442 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:31.420051 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:31.420798 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:31.420950 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:31.426734 (+ 5784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:15:31.426756 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:15:31.426776 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:15:31.426828 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:15:31.430089 (+ 3261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:31.431072 (+ 983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:31.431081 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:31.431084 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:31.431240 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:31.431668 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:31.431673 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:31.431675 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:31.432152 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:15:31.432402 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:31.432802 (+ 400us) server_negotiation.cc:299] Negotiation successful
0811 19:15:31.433013 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260811 19:15:31.434062 30082 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:15:31.529523 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 29882
I20260811 19:15:31.560472 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 30097
Aug 11 19:15:31 dist-test-slave-t7g5 krb5kdc[29479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475727, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.131@KRBTEST.COM
I20260811 19:15:31.578990 30338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:31.561503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54084 (local address 127.3.61.131:37193)
0811 19:15:31.561899 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:31.561904 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:31.561924 (+ 20us) server_negotiation.cc:407] Connection header received
0811 19:15:31.561985 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:31.561988 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:31.562081 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:31.562176 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:15:31.563600 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:31.564290 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:31.565876 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:31.566318 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:31.566436 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:31.571960 (+ 5524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:15:31.572005 (+ 45us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:15:31.572030 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:15:31.572091 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:15:31.575866 (+ 3775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:31.576776 (+ 910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:31.576785 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:31.576788 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:31.576950 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:31.577336 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:31.577340 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:31.577342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:31.577791 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:15:31.578108 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:31.578421 (+ 313us) server_negotiation.cc:299] Negotiation successful
0811 19:15:31.578659 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260811 19:15:31.579892 30298 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:15:31.679284 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 30097
I20260811 19:15:31.710117 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 29501
I20260811 19:15:31.711331 29598 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:15:31.837630 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 29501
I20260811 19:15:51.869688 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:43283
--webserver_interface=127.3.61.190
--webserver_port=42311
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:35451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.61.190:43283
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:15:52.259882 30401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:15:52.260285 30401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:15:52.260448 30401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:15:52.269090 30401 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 19:15:52.269289 30401 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 19:15:52.269338 30401 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 19:15:52.269426 30401 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:15:52.269517 30401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:15:52.269635 30401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:15:52.269750 30401 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:15:52.269840 30401 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:15:52.281553 30401 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:35451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.3.61.190:43283
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:43283
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.61.190
--webserver_port=42311
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:15:52.283094 30401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:15:52.284607 30401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:15:52.314608 30410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:15:52.314604 30411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:15:52.314641 30413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:15:52.315054 30401 server_base.cc:1034] running on GCE node
I20260811 19:15:52.316972 30401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:15:52.319533 30401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:15:52.320892 30401 hybrid_clock.cc:648] HybridClock initialized: now 1786475752320787 us; error 62 us; skew 500 ppm
I20260811 19:15:52.321444 30401 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:15:52 dist-test-slave-t7g5 krb5kdc[29479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475752, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:15:52.328819 30401 init.cc:377] Logged in from keytab as kudu/127.3.61.190@KRBTEST.COM (short username kudu)
I20260811 19:15:52.330850 30401 webserver.cc:460] Webserver started at http://127.3.61.190:42311/ using document root <none> and password file <none>
I20260811 19:15:52.331710 30401 fs_manager.cc:359] Metadata directory not provided
I20260811 19:15:52.331845 30401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:15:52.340149 30401 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.002s sys 0.003s
I20260811 19:15:52.344151 30423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:15:52.345880 30401 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:15:52.346086 30401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/master-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/master-0/wal
uuid: "41c09bf6448a4760973e8a7897b4ccd0"
format_stamp: "Formatted at 2026-08-11 19:15:28 on dist-test-slave-t7g5"
server_key: "3d241892a73f429e804e9ec3331fb46e"
server_key_iv: "a47e4b8a88ff5ed8f5c0e2cf7e78072e"
server_key_version: "encryptionkey@0"
I20260811 19:15:52.347225 30401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:15:52.363348 30401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:15:52.368561 30401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:15:52.368892 30401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:15:52.390177 30401 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.190:43283
I20260811 19:15:52.390221 30508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.190:43283 every 8 connection(s)
I20260811 19:15:52.391919 30401 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
I20260811 19:15:52.397336 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 30401
I20260811 19:15:52.398218 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.129:41967
--local_ip_for_outbound_sockets=127.3.61.129
--tserver_master_addrs=127.3.61.190:43283
--webserver_port=44995
--webserver_interface=127.3.61.129
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab
--principal=kudu/127.3.61.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:35451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:15:52.400167 30509 sys_catalog.cc:263] Verifying existing consensus state
I20260811 19:15:52.403913 30509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0: Bootstrap starting.
I20260811 19:15:52.426215 30509 log.cc:824] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0: Log is configured to *not* fsync() on all Append() calls
I20260811 19:15:52.443579 30509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 19:15:52.444222 30509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0: Bootstrap complete.
I20260811 19:15:52.454736 30509 raft_consensus.cc:348] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41c09bf6448a4760973e8a7897b4ccd0" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 43283 } }
I20260811 19:15:52.456229 30509 raft_consensus.cc:729] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41c09bf6448a4760973e8a7897b4ccd0, State: Initialized, Role: FOLLOWER
I20260811 19:15:52.457131 30509 consensus_queue.cc:260] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41c09bf6448a4760973e8a7897b4ccd0" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 43283 } }
I20260811 19:15:52.457437 30509 raft_consensus.cc:388] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:15:52.457754 30509 raft_consensus.cc:482] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:15:52.458123 30509 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0 [term 1 FOLLOWER]: Advancing to term 2
I20260811 19:15:52.463498 30509 raft_consensus.cc:504] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41c09bf6448a4760973e8a7897b4ccd0" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 43283 } }
I20260811 19:15:52.464414 30509 leader_election.cc:302] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 41c09bf6448a4760973e8a7897b4ccd0; no voters:
I20260811 19:15:52.465432 30509 leader_election.cc:288] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 19:15:52.465621 30515 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 19:15:52.466882 30515 raft_consensus.cc:686] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0 [term 2 LEADER]: Becoming Leader. State: Replica: 41c09bf6448a4760973e8a7897b4ccd0, State: Running, Role: LEADER
I20260811 19:15:52.467581 30515 consensus_queue.cc:237] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41c09bf6448a4760973e8a7897b4ccd0" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 43283 } }
I20260811 19:15:52.468111 30509 sys_catalog.cc:564] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:15:52.470345 30517 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 41c09bf6448a4760973e8a7897b4ccd0. Latest consensus state: current_term: 2 leader_uuid: "41c09bf6448a4760973e8a7897b4ccd0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41c09bf6448a4760973e8a7897b4ccd0" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 43283 } } }
I20260811 19:15:52.470819 30517 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0 [sys.catalog]: This master's current role is: LEADER
I20260811 19:15:52.471429 30516 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "41c09bf6448a4760973e8a7897b4ccd0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41c09bf6448a4760973e8a7897b4ccd0" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 43283 } } }
I20260811 19:15:52.471799 30516 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0 [sys.catalog]: This master's current role is: LEADER
I20260811 19:15:52.483604 30526 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:15:52.492776 30526 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:15:52.494833 30526 catalog_manager.cc:1259] Loaded cluster ID: 3f91653b5f7b4343ad108648ffeec1e3
I20260811 19:15:52.495077 30526 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:15:52.503252 30526 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:15:52.505475 30526 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0: Loaded TSK: 0
I20260811 19:15:52.552563 30526 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0: Generated new TSK 1
I20260811 19:15:52.553051 30526 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 19:15:52.832033 30511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:15:52.832373 30511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:15:52.832515 30511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:15:52.841881 30511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:15:52.842108 30511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:15:52.842298 30511 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.129
I20260811 19:15:52.854380 30511 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:35451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab
--principal=kudu/127.3.61.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.129:41967
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.61.129
--webserver_port=44995
--tserver_master_addrs=127.3.61.190:43283
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.129
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:15:52.855858 30511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:15:52.857234 30511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:15:52.873790 30554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:15:52.873924 30553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:15:52.874565 30557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:15:52.875717 30511 server_base.cc:1034] running on GCE node
I20260811 19:15:52.876430 30511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:15:52.877800 30511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:15:52.879144 30511 hybrid_clock.cc:648] HybridClock initialized: now 1786475752879060 us; error 65 us; skew 500 ppm
I20260811 19:15:52.879555 30511 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:15:52 dist-test-slave-t7g5 krb5kdc[29479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475752, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:15:52.883268 30511 init.cc:377] Logged in from keytab as kudu/127.3.61.129@KRBTEST.COM (short username kudu)
I20260811 19:15:52.885571 30511 webserver.cc:460] Webserver started at http://127.3.61.129:44995/ using document root <none> and password file <none>
I20260811 19:15:52.886612 30511 fs_manager.cc:359] Metadata directory not provided
I20260811 19:15:52.886781 30511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:15:52.893810 30511 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 19:15:52.897289 30571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:15:52.899246 30511 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 19:15:52.899449 30511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-0/wal
uuid: "4b8d3747e7a240acb769d8b40c9fff76"
format_stamp: "Formatted at 2026-08-11 19:15:29 on dist-test-slave-t7g5"
server_key: "e7f851f8670b06e7bf3d0cd48f0abe3a"
server_key_iv: "f9cbae46258a0c9ac2c14426e0da52d4"
server_key_version: "encryptionkey@0"
I20260811 19:15:52.900823 30511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:15:52.924283 30511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:15:52.929106 30511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:15:52.929392 30511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:15:52.930564 30511 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:15:52.932237 30511 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:15:52.932343 30511 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:15:52.932458 30511 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:15:52.932529 30511 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:15:52.966410 30511 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.129:41967
I20260811 19:15:52.966499 30765 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.129:41967 every 8 connection(s)
I20260811 19:15:52.968021 30511 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
I20260811 19:15:52.973228 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 30511
I20260811 19:15:52.973961 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.130:41047
--local_ip_for_outbound_sockets=127.3.61.130
--tserver_master_addrs=127.3.61.190:43283
--webserver_port=43429
--webserver_interface=127.3.61.130
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab
--principal=kudu/127.3.61.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:35451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 19:15:52 dist-test-slave-t7g5 krb5kdc[29479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475752, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.129@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:15:52.998816 30772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:52.972231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43283 (local address 127.3.61.129:48551)
0811 19:15:52.973109 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:52.973165 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 19:15:52.975672 (+ 2507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:15:52.977769 (+ 2097us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:15:52.977779 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:15:52.978363 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:15:52.979080 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:52.979105 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:52.981427 (+ 2322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:52.981434 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:52.983329 (+ 1895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:52.983353 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:52.984950 (+ 1597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:52.984958 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:52.985192 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:52.985965 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:15:52.985995 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:15:52.988589 (+ 2594us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:15:52.993351 (+ 4762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:15:52.993361 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:15:52.993375 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:15:52.993986 (+ 611us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:15:52.994881 (+ 895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:15:52.994885 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:15:52.994887 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:15:52.995224 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:15:52.996441 (+ 1217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:15:52.996448 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:15:52.996801 (+ 353us) client_negotiation.cc:769] Sending connection context
0811 19:15:52.997052 (+ 251us) client_negotiation.cc:240] Negotiation successful
0811 19:15:52.997508 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":343,"threads_started":1}
I20260811 19:15:53.000406 30771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:52.972026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:48551 (local address 127.3.61.190:43283)
0811 19:15:52.972703 (+ 677us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:52.972720 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:52.973325 (+ 605us) server_negotiation.cc:407] Connection header received
0811 19:15:52.976501 (+ 3176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:52.976524 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:52.977012 (+ 488us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:52.977492 (+ 480us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:15:52.979323 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:52.981093 (+ 1770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:52.983564 (+ 2471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:52.984615 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:52.984829 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:52.989069 (+ 4240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:15:52.989081 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:15:52.989105 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:15:52.989160 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:15:52.993112 (+ 3952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:52.994271 (+ 1159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:52.994285 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:52.994296 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:52.994694 (+ 398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:52.995395 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:52.995402 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:52.995405 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:52.996262 (+ 857us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:15:52.996485 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:52.997501 (+ 1016us) server_negotiation.cc:299] Negotiation successful
0811 19:15:52.998325 (+ 824us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":345,"threads_started":1}
I20260811 19:15:53.003609 30766 heartbeater.cc:347] Connected to a master server at 127.3.61.190:43283
I20260811 19:15:53.003957 30766 heartbeater.cc:464] Registering TS with master...
I20260811 19:15:53.004825 30766 heartbeater.cc:511] Master 127.3.61.190:43283 requested a full tablet report, sending...
I20260811 19:15:53.007939 30447 ts_manager.cc:194] Registered new tserver with Master: 4b8d3747e7a240acb769d8b40c9fff76 (127.3.61.129:41967)
I20260811 19:15:53.009989 30447 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.129@KRBTEST.COM'} at 127.3.61.129:48551
W20260811 19:15:53.361052 30773 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:15:53.361443 30773 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:15:53.361568 30773 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:15:53.369779 30773 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:15:53.369951 30773 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:15:53.370256 30773 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.130
I20260811 19:15:53.382231 30773 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:35451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab
--principal=kudu/127.3.61.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.130:41047
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.61.130
--webserver_port=43429
--tserver_master_addrs=127.3.61.190:43283
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.130
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:15:53.383718 30773 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:15:53.385139 30773 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:15:53.399878 30784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:15:53.400333 30788 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:15:53.400406 30783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:15:53.407557 30773 server_base.cc:1034] running on GCE node
I20260811 19:15:53.408311 30773 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:15:53.409622 30773 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:15:53.410923 30773 hybrid_clock.cc:648] HybridClock initialized: now 1786475753410855 us; error 54 us; skew 500 ppm
I20260811 19:15:53.411273 30773 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:15:53 dist-test-slave-t7g5 krb5kdc[29479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475753, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:15:53.414994 30773 init.cc:377] Logged in from keytab as kudu/127.3.61.130@KRBTEST.COM (short username kudu)
I20260811 19:15:53.416864 30773 webserver.cc:460] Webserver started at http://127.3.61.130:43429/ using document root <none> and password file <none>
I20260811 19:15:53.417708 30773 fs_manager.cc:359] Metadata directory not provided
I20260811 19:15:53.417856 30773 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:15:53.424264 30773 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 19:15:53.428172 30801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:15:53.429909 30773 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 19:15:53.430122 30773 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-1/wal
uuid: "4a86881f7b5949c6b4c66b5501d511a6"
format_stamp: "Formatted at 2026-08-11 19:15:30 on dist-test-slave-t7g5"
server_key: "280d131b916c7795b0661227a79df126"
server_key_iv: "8cd44d4237f74d5e912f97328baed288"
server_key_version: "encryptionkey@0"
I20260811 19:15:53.431228 30773 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:15:53.453181 30773 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:15:53.459170 30773 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:15:53.459479 30773 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:15:53.460884 30773 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:15:53.462476 30773 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:15:53.462579 30773 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:15:53.462703 30773 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:15:53.462777 30773 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:15:53.501926 30773 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.130:41047
I20260811 19:15:53.502066 30973 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.130:41047 every 8 connection(s)
I20260811 19:15:53.503604 30773 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
I20260811 19:15:53.507805 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 30773
I20260811 19:15:53.508610 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.131:37193
--local_ip_for_outbound_sockets=127.3.61.131
--tserver_master_addrs=127.3.61.190:43283
--webserver_port=36513
--webserver_interface=127.3.61.131
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab
--principal=kudu/127.3.61.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:35451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 19:15:53 dist-test-slave-t7g5 krb5kdc[29479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475753, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.130@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:15:53.528479 30979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:53.506608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:50983 (local address 127.3.61.190:43283)
0811 19:15:53.507145 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:53.507151 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:53.508488 (+ 1337us) server_negotiation.cc:407] Connection header received
0811 19:15:53.511010 (+ 2522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:53.511017 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:53.511145 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:53.511270 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:15:53.513392 (+ 2122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:53.514306 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:53.516154 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:53.516633 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:53.516761 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:53.521174 (+ 4413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:15:53.521181 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:15:53.521184 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:15:53.521228 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:15:53.524580 (+ 3352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:53.525704 (+ 1124us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:53.525711 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:53.525714 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:53.525876 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:53.526449 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:53.526454 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:53.526456 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:53.526786 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:15:53.526909 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:53.527942 (+ 1033us) server_negotiation.cc:299] Negotiation successful
0811 19:15:53.528178 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":280,"threads_started":1}
I20260811 19:15:53.529968 30980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:53.507069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43283 (local address 127.3.61.130:50983)
0811 19:15:53.508236 (+ 1167us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:53.508310 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0811 19:15:53.510609 (+ 2299us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:15:53.511636 (+ 1027us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:15:53.511648 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:15:53.512393 (+ 745us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:15:53.513174 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:53.513198 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:53.514491 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:53.514496 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:53.515989 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:53.516004 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:53.516925 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:53.516933 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:53.517214 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:53.518270 (+ 1056us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:15:53.518315 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:15:53.520798 (+ 2483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:15:53.524837 (+ 4039us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:15:53.524849 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:15:53.524871 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:15:53.525516 (+ 645us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:15:53.526029 (+ 513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:15:53.526035 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:15:53.526038 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:15:53.526306 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:15:53.526952 (+ 646us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:15:53.526961 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:15:53.527507 (+ 546us) client_negotiation.cc:769] Sending connection context
0811 19:15:53.527840 (+ 333us) client_negotiation.cc:240] Negotiation successful
0811 19:15:53.528434 (+ 594us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":680,"thread_start_us":427,"threads_started":1}
I20260811 19:15:53.531756 30976 heartbeater.cc:347] Connected to a master server at 127.3.61.190:43283
I20260811 19:15:53.532099 30976 heartbeater.cc:464] Registering TS with master...
I20260811 19:15:53.532938 30976 heartbeater.cc:511] Master 127.3.61.190:43283 requested a full tablet report, sending...
I20260811 19:15:53.535380 30447 ts_manager.cc:194] Registered new tserver with Master: 4a86881f7b5949c6b4c66b5501d511a6 (127.3.61.130:41047)
I20260811 19:15:53.536552 30447 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.130@KRBTEST.COM'} at 127.3.61.130:50983
W20260811 19:15:53.890269 30981 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:15:53.890607 30981 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:15:53.890713 30981 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:15:53.898636 30981 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:15:53.898792 30981 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:15:53.898916 30981 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.131
I20260811 19:15:53.911612 30981 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:35451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab
--principal=kudu/127.3.61.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.131:37193
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.61.131
--webserver_port=36513
--tserver_master_addrs=127.3.61.190:43283
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.131
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:15:53.913028 30981 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:15:53.914487 30981 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:15:53.928864 31002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:15:53.928845 30999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:15:53.929092 30997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:15:53.929893 30981 server_base.cc:1034] running on GCE node
I20260811 19:15:53.930575 30981 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:15:53.931671 30981 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:15:53.932987 30981 hybrid_clock.cc:648] HybridClock initialized: now 1786475753932923 us; error 58 us; skew 500 ppm
I20260811 19:15:53.933359 30981 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:15:53 dist-test-slave-t7g5 krb5kdc[29479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475753, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:15:53.937410 30981 init.cc:377] Logged in from keytab as kudu/127.3.61.131@KRBTEST.COM (short username kudu)
I20260811 19:15:53.939473 30981 webserver.cc:460] Webserver started at http://127.3.61.131:36513/ using document root <none> and password file <none>
I20260811 19:15:53.940382 30981 fs_manager.cc:359] Metadata directory not provided
I20260811 19:15:53.940547 30981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:15:53.947847 30981 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 19:15:53.951448 31013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:15:53.953154 30981 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 19:15:53.953343 30981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-2/wal
uuid: "0b1650603d4846b0ab3a75db70545cd9"
format_stamp: "Formatted at 2026-08-11 19:15:30 on dist-test-slave-t7g5"
server_key: "57563b6588e292f0c6dac01ca95bbbae"
server_key_iv: "9303a5f0bde5aa3283ddcc2dad2497a8"
server_key_version: "encryptionkey@0"
I20260811 19:15:53.954998 30981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:15:53.975989 30981 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:15:53.981911 30981 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:15:53.982263 30981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:15:53.983527 30981 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:15:53.985350 30981 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:15:53.985504 30981 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:15:53.985637 30981 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:15:53.985751 30981 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260811 19:15:54.013736 30766 heartbeater.cc:503] Master 127.3.61.190:43283 was elected leader, sending a full tablet report...
I20260811 19:15:54.022863 30981 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.131:37193
I20260811 19:15:54.022912 31208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.131:37193 every 8 connection(s)
I20260811 19:15:54.024410 30981 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
I20260811 19:15:54.030171 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 30981
Aug 11 19:15:54 dist-test-slave-t7g5 krb5kdc[29479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475753, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.131@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:15:54.052348 31218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:54.031300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56112 (local address 127.3.61.190:43283)
0811 19:15:54.031834 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:54.031839 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:54.032413 (+ 574us) server_negotiation.cc:407] Connection header received
0811 19:15:54.032559 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:54.032625 (+ 66us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:54.032703 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:54.032796 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:15:54.034942 (+ 2146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:54.036440 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:54.038599 (+ 2159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:54.039628 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:54.039890 (+ 262us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:54.043782 (+ 3892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:15:54.043788 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:15:54.043793 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:15:54.043842 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:15:54.048124 (+ 4282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:54.049481 (+ 1357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:54.049486 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:54.049488 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:54.049732 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:54.050512 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:54.050516 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:54.050519 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:54.051004 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:15:54.051271 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:54.051708 (+ 437us) server_negotiation.cc:299] Negotiation successful
0811 19:15:54.051949 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":158,"server-negotiator.queue_time_us":413,"spinlock_wait_cycles":14208,"thread_start_us":259,"threads_started":1}
I20260811 19:15:54.054452 30979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:54.027790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:36209 (local address 127.3.61.190:43283)
0811 19:15:54.028092 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:54.028097 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:54.029278 (+ 1181us) server_negotiation.cc:407] Connection header received
0811 19:15:54.031849 (+ 2571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:54.031856 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:54.031972 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:54.032089 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:15:54.034613 (+ 2524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:54.036147 (+ 1534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:54.038787 (+ 2640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:54.039628 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:54.039894 (+ 266us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:54.044526 (+ 4632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:15:54.044533 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:15:54.044538 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:15:54.044579 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:15:54.048411 (+ 3832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:54.049592 (+ 1181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:54.049597 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:54.049601 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:54.049780 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:54.050692 (+ 912us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:54.050697 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:54.050700 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:54.051056 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:15:54.051171 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:54.053870 (+ 2699us) server_negotiation.cc:299] Negotiation successful
0811 19:15:54.054112 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":123,"server-negotiator.queue_time_us":163,"spinlock_wait_cycles":1408}
I20260811 19:15:54.055122 31214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:54.028233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43283 (local address 127.3.61.131:36209)
0811 19:15:54.029094 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:54.029136 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 19:15:54.031455 (+ 2319us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:15:54.032430 (+ 975us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:15:54.032445 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:15:54.033249 (+ 804us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:15:54.034239 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:54.034270 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:54.036382 (+ 2112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:54.036387 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:54.038617 (+ 2230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:54.038633 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:54.040039 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:54.040047 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:54.040386 (+ 339us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:54.041176 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:15:54.041225 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:15:54.044239 (+ 3014us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:15:54.048716 (+ 4477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:15:54.048729 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:15:54.048751 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:15:54.049447 (+ 696us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:15:54.049903 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:15:54.049908 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:15:54.049911 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:15:54.050461 (+ 550us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:15:54.051196 (+ 735us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:15:54.051206 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:15:54.051794 (+ 588us) client_negotiation.cc:769] Sending connection context
0811 19:15:54.052176 (+ 382us) client_negotiation.cc:240] Negotiation successful
0811 19:15:54.052782 (+ 606us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":342,"threads_started":1}
I20260811 19:15:54.056923 31210 heartbeater.cc:347] Connected to a master server at 127.3.61.190:43283
I20260811 19:15:54.057322 31210 heartbeater.cc:464] Registering TS with master...
I20260811 19:15:54.058579 31210 heartbeater.cc:511] Master 127.3.61.190:43283 requested a full tablet report, sending...
I20260811 19:15:54.060992 30447 ts_manager.cc:194] Registered new tserver with Master: 0b1650603d4846b0ab3a75db70545cd9 (127.3.61.131:37193)
I20260811 19:15:54.061951 30447 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.131@KRBTEST.COM'} at 127.3.61.131:36209
I20260811 19:15:54.069135 3318 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 19:15:54.076738 30979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:54.070375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56116 (local address 127.3.61.190:43283)
0811 19:15:54.070622 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:54.070628 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:54.070926 (+ 298us) server_negotiation.cc:407] Connection header received
0811 19:15:54.071137 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:54.071141 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:54.071299 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:54.071446 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:15:54.072049 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:54.072945 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:54.074631 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:54.075124 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:54.075238 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:54.075826 (+ 588us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:15:54.076130 (+ 304us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0811 19:15:54.076490 (+ 360us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56116: token expired
Metrics: {"server-negotiator.queue_time_us":79}
W20260811 19:15:54.076861 30979 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56116: token expired
W20260811 19:15:54.078097 30318 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.3.61.190:43283)
I20260811 19:15:54.093604 30979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:54.078777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56120 (local address 127.3.61.190:43283)
0811 19:15:54.078934 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:54.078938 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:54.078956 (+ 18us) server_negotiation.cc:407] Connection header received
0811 19:15:54.079169 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:54.079173 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:54.079241 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:54.079338 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:15:54.080893 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:54.081772 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:54.083335 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:54.083774 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:54.083875 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:54.086913 (+ 3038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:15:54.086919 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:15:54.086923 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:15:54.087008 (+ 85us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:15:54.090225 (+ 3217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:54.091151 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:54.091156 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:54.091158 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:54.091309 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:54.091823 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:54.091830 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:54.091832 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:54.092314 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:15:54.092619 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:54.093051 (+ 432us) server_negotiation.cc:299] Negotiation successful
0811 19:15:54.093306 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 19:15:54.098181 30447 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56120:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 19:15:54.105795 30447 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:15:54.148429 31225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:54.121398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33248 (local address 127.3.61.131:37193)
0811 19:15:54.139640 (+ 18242us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:54.139649 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:54.139680 (+ 31us) server_negotiation.cc:407] Connection header received
0811 19:15:54.139791 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:54.139798 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:54.139999 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:54.140975 (+ 976us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:15:54.141591 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:54.142944 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:54.145293 (+ 2349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:54.146661 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:54.146843 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:54.147125 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:54.147849 (+ 724us) server_negotiation.cc:299] Negotiation successful
0811 19:15:54.148059 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18060,"thread_start_us":17874,"threads_started":1}
I20260811 19:15:54.148587 31226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:54.121232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.131:37193 (local address 127.0.0.1:33248)
0811 19:15:54.122645 (+ 1413us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:54.122695 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 19:15:54.122931 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:15:54.140281 (+ 17350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:15:54.140285 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:15:54.140313 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:15:54.140753 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:54.140765 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:54.143164 (+ 2399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:54.143170 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:54.145086 (+ 1916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:54.145106 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:54.147316 (+ 2210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:54.147322 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:54.147563 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:54.147623 (+ 60us) client_negotiation.cc:769] Sending connection context
0811 19:15:54.147764 (+ 141us) client_negotiation.cc:240] Negotiation successful
0811 19:15:54.148037 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1094,"mutex_wait_us":37,"thread_start_us":919,"threads_started":1}
I20260811 19:15:54.148702 31234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:54.121751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:41967 (local address 127.0.0.1:55090)
0811 19:15:54.131134 (+ 9383us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:54.131162 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 19:15:54.131346 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:15:54.138630 (+ 7284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:15:54.138639 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:15:54.138712 (+ 73us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:15:54.139235 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:54.139252 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:54.141880 (+ 2628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:54.141885 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:54.145133 (+ 3248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:54.145156 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:54.147734 (+ 2578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:54.147742 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:54.147909 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:54.147978 (+ 69us) client_negotiation.cc:769] Sending connection context
0811 19:15:54.148061 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 19:15:54.148208 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9192,"mutex_wait_us":309,"spinlock_wait_cycles":24832,"thread_start_us":17079,"threads_started":1}
I20260811 19:15:54.149744 31232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:54.124050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60272 (local address 127.3.61.130:41047)
0811 19:15:54.134841 (+ 10791us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:54.134852 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:54.138851 (+ 3999us) server_negotiation.cc:407] Connection header received
0811 19:15:54.139010 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:54.139020 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:54.139267 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:54.139479 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:15:54.140281 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:54.141628 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:54.145374 (+ 3746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:54.148384 (+ 3010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:54.148527 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:54.148733 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:54.149299 (+ 566us) server_negotiation.cc:299] Negotiation successful
0811 19:15:54.149445 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10570,"thread_start_us":10304,"threads_started":1}
I20260811 19:15:54.149742 31227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:54.123309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.130:41047 (local address 127.0.0.1:60272)
0811 19:15:54.138497 (+ 15188us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:54.138527 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 19:15:54.138704 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:15:54.139639 (+ 935us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:15:54.139644 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:15:54.139671 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:15:54.139975 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:54.139987 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:54.141805 (+ 1818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:54.141811 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:54.145133 (+ 3322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:54.145156 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:54.148920 (+ 3764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:54.148927 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:54.149090 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:54.149122 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 19:15:54.149229 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 19:15:54.149366 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15016,"mutex_wait_us":301,"spinlock_wait_cycles":2432,"thread_start_us":7411,"threads_started":1}
I20260811 19:15:54.150748 31230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:54.122559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55090 (local address 127.3.61.129:41967)
0811 19:15:54.137982 (+ 15423us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:54.137989 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:54.138018 (+ 29us) server_negotiation.cc:407] Connection header received
0811 19:15:54.138117 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:54.138123 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:54.138356 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:54.138584 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:15:54.139487 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:54.141642 (+ 2155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:54.145391 (+ 3749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:54.147427 (+ 2036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:54.148952 (+ 1525us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:54.149770 (+ 818us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:54.150163 (+ 393us) server_negotiation.cc:299] Negotiation successful
0811 19:15:54.150382 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15260,"thread_start_us":15120,"threads_started":1}
I20260811 19:15:54.153987 30868 tablet_service.cc:1467] Processing CreateTablet for tablet f2540d657eea4171b930aa7bc8e79bcb (DEFAULT_TABLE table=test-table [id=2ed3889ac2364292bd89a70eca0d70b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:15:54.155539 30868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2540d657eea4171b930aa7bc8e79bcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:15:54.154912 31091 tablet_service.cc:1467] Processing CreateTablet for tablet f2540d657eea4171b930aa7bc8e79bcb (DEFAULT_TABLE table=test-table [id=2ed3889ac2364292bd89a70eca0d70b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:15:54.157234 31091 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2540d657eea4171b930aa7bc8e79bcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:15:54.156910 30656 tablet_service.cc:1467] Processing CreateTablet for tablet f2540d657eea4171b930aa7bc8e79bcb (DEFAULT_TABLE table=test-table [id=2ed3889ac2364292bd89a70eca0d70b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:15:54.158660 30656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2540d657eea4171b930aa7bc8e79bcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:15:54.171028 31236 tablet_bootstrap.cc:492] T f2540d657eea4171b930aa7bc8e79bcb P 4a86881f7b5949c6b4c66b5501d511a6: Bootstrap starting.
I20260811 19:15:54.175907 31236 tablet_bootstrap.cc:654] T f2540d657eea4171b930aa7bc8e79bcb P 4a86881f7b5949c6b4c66b5501d511a6: Neither blocks nor log segments found. Creating new log.
I20260811 19:15:54.177734 31236 log.cc:824] T f2540d657eea4171b930aa7bc8e79bcb P 4a86881f7b5949c6b4c66b5501d511a6: Log is configured to *not* fsync() on all Append() calls
I20260811 19:15:54.178068 31239 tablet_bootstrap.cc:492] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76: Bootstrap starting.
I20260811 19:15:54.182647 31238 tablet_bootstrap.cc:492] T f2540d657eea4171b930aa7bc8e79bcb P 0b1650603d4846b0ab3a75db70545cd9: Bootstrap starting.
I20260811 19:15:54.183531 31239 tablet_bootstrap.cc:654] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76: Neither blocks nor log segments found. Creating new log.
I20260811 19:15:54.184821 31239 log.cc:824] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76: Log is configured to *not* fsync() on all Append() calls
I20260811 19:15:54.187194 31238 tablet_bootstrap.cc:654] T f2540d657eea4171b930aa7bc8e79bcb P 0b1650603d4846b0ab3a75db70545cd9: Neither blocks nor log segments found. Creating new log.
I20260811 19:15:54.187418 31236 tablet_bootstrap.cc:492] T f2540d657eea4171b930aa7bc8e79bcb P 4a86881f7b5949c6b4c66b5501d511a6: No bootstrap required, opened a new log
I20260811 19:15:54.187791 31236 ts_tablet_manager.cc:1397] T f2540d657eea4171b930aa7bc8e79bcb P 4a86881f7b5949c6b4c66b5501d511a6: Time spent bootstrapping tablet: real 0.017s user 0.006s sys 0.008s
I20260811 19:15:54.191064 31239 tablet_bootstrap.cc:492] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76: No bootstrap required, opened a new log
I20260811 19:15:54.191398 31239 ts_tablet_manager.cc:1397] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.007s
I20260811 19:15:54.196329 31238 log.cc:824] T f2540d657eea4171b930aa7bc8e79bcb P 0b1650603d4846b0ab3a75db70545cd9: Log is configured to *not* fsync() on all Append() calls
I20260811 19:15:54.200915 31236 raft_consensus.cc:348] T f2540d657eea4171b930aa7bc8e79bcb P 4a86881f7b5949c6b4c66b5501d511a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b1650603d4846b0ab3a75db70545cd9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37193 } } peers { permanent_uuid: "4a86881f7b5949c6b4c66b5501d511a6" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41047 } } peers { permanent_uuid: "4b8d3747e7a240acb769d8b40c9fff76" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 41967 } }
I20260811 19:15:54.201452 31236 raft_consensus.cc:374] T f2540d657eea4171b930aa7bc8e79bcb P 4a86881f7b5949c6b4c66b5501d511a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:15:54.201563 31236 raft_consensus.cc:729] T f2540d657eea4171b930aa7bc8e79bcb P 4a86881f7b5949c6b4c66b5501d511a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a86881f7b5949c6b4c66b5501d511a6, State: Initialized, Role: FOLLOWER
I20260811 19:15:54.202325 31236 consensus_queue.cc:260] T f2540d657eea4171b930aa7bc8e79bcb P 4a86881f7b5949c6b4c66b5501d511a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b1650603d4846b0ab3a75db70545cd9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37193 } } peers { permanent_uuid: "4a86881f7b5949c6b4c66b5501d511a6" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41047 } } peers { permanent_uuid: "4b8d3747e7a240acb769d8b40c9fff76" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 41967 } }
I20260811 19:15:54.203999 31238 tablet_bootstrap.cc:492] T f2540d657eea4171b930aa7bc8e79bcb P 0b1650603d4846b0ab3a75db70545cd9: No bootstrap required, opened a new log
I20260811 19:15:54.204290 31238 ts_tablet_manager.cc:1397] T f2540d657eea4171b930aa7bc8e79bcb P 0b1650603d4846b0ab3a75db70545cd9: Time spent bootstrapping tablet: real 0.022s user 0.006s sys 0.005s
I20260811 19:15:54.205608 30976 heartbeater.cc:503] Master 127.3.61.190:43283 was elected leader, sending a full tablet report...
I20260811 19:15:54.206355 31236 ts_tablet_manager.cc:1428] T f2540d657eea4171b930aa7bc8e79bcb P 4a86881f7b5949c6b4c66b5501d511a6: Time spent starting tablet: real 0.018s user 0.015s sys 0.001s
I20260811 19:15:54.204494 31239 raft_consensus.cc:348] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b1650603d4846b0ab3a75db70545cd9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37193 } } peers { permanent_uuid: "4a86881f7b5949c6b4c66b5501d511a6" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41047 } } peers { permanent_uuid: "4b8d3747e7a240acb769d8b40c9fff76" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 41967 } }
I20260811 19:15:54.206754 31239 raft_consensus.cc:374] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:15:54.206867 31239 raft_consensus.cc:729] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b8d3747e7a240acb769d8b40c9fff76, State: Initialized, Role: FOLLOWER
I20260811 19:15:54.208567 31239 consensus_queue.cc:260] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b1650603d4846b0ab3a75db70545cd9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37193 } } peers { permanent_uuid: "4a86881f7b5949c6b4c66b5501d511a6" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41047 } } peers { permanent_uuid: "4b8d3747e7a240acb769d8b40c9fff76" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 41967 } }
I20260811 19:15:54.210566 31239 ts_tablet_manager.cc:1428] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76: Time spent starting tablet: real 0.019s user 0.006s sys 0.011s
I20260811 19:15:54.215075 31238 raft_consensus.cc:348] T f2540d657eea4171b930aa7bc8e79bcb P 0b1650603d4846b0ab3a75db70545cd9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b1650603d4846b0ab3a75db70545cd9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37193 } } peers { permanent_uuid: "4a86881f7b5949c6b4c66b5501d511a6" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41047 } } peers { permanent_uuid: "4b8d3747e7a240acb769d8b40c9fff76" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 41967 } }
I20260811 19:15:54.215533 31238 raft_consensus.cc:374] T f2540d657eea4171b930aa7bc8e79bcb P 0b1650603d4846b0ab3a75db70545cd9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:15:54.215642 31238 raft_consensus.cc:729] T f2540d657eea4171b930aa7bc8e79bcb P 0b1650603d4846b0ab3a75db70545cd9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b1650603d4846b0ab3a75db70545cd9, State: Initialized, Role: FOLLOWER
I20260811 19:15:54.216234 31238 consensus_queue.cc:260] T f2540d657eea4171b930aa7bc8e79bcb P 0b1650603d4846b0ab3a75db70545cd9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b1650603d4846b0ab3a75db70545cd9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37193 } } peers { permanent_uuid: "4a86881f7b5949c6b4c66b5501d511a6" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41047 } } peers { permanent_uuid: "4b8d3747e7a240acb769d8b40c9fff76" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 41967 } }
I20260811 19:15:54.217551 31210 heartbeater.cc:503] Master 127.3.61.190:43283 was elected leader, sending a full tablet report...
I20260811 19:15:54.217864 31238 ts_tablet_manager.cc:1428] T f2540d657eea4171b930aa7bc8e79bcb P 0b1650603d4846b0ab3a75db70545cd9: Time spent starting tablet: real 0.013s user 0.001s sys 0.013s
W20260811 19:15:54.221306 30767 tablet.cc:2369] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 19:15:54.256070 30978 tablet.cc:2369] T f2540d657eea4171b930aa7bc8e79bcb P 4a86881f7b5949c6b4c66b5501d511a6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 19:15:54.276274 31211 tablet.cc:2369] T f2540d657eea4171b930aa7bc8e79bcb P 0b1650603d4846b0ab3a75db70545cd9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:15:54.377017 31244 raft_consensus.cc:482] T f2540d657eea4171b930aa7bc8e79bcb P 4a86881f7b5949c6b4c66b5501d511a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:15:54.377451 31244 raft_consensus.cc:504] T f2540d657eea4171b930aa7bc8e79bcb P 4a86881f7b5949c6b4c66b5501d511a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b1650603d4846b0ab3a75db70545cd9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37193 } } peers { permanent_uuid: "4a86881f7b5949c6b4c66b5501d511a6" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41047 } } peers { permanent_uuid: "4b8d3747e7a240acb769d8b40c9fff76" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 41967 } }
I20260811 19:15:54.379992 31244 leader_election.cc:288] T f2540d657eea4171b930aa7bc8e79bcb P 4a86881f7b5949c6b4c66b5501d511a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0b1650603d4846b0ab3a75db70545cd9 (127.3.61.131:37193), 4b8d3747e7a240acb769d8b40c9fff76 (127.3.61.129:41967)
I20260811 19:15:54.390260 31225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:54.380309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:43735 (local address 127.3.61.131:37193)
0811 19:15:54.380534 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:54.380540 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:54.381217 (+ 677us) server_negotiation.cc:407] Connection header received
0811 19:15:54.381455 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:54.381461 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:54.381587 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:54.381712 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:15:54.382651 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:54.383887 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:54.387252 (+ 3365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:54.388607 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:54.388821 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:54.388902 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:54.389852 (+ 950us) server_negotiation.cc:299] Negotiation successful
0811 19:15:54.390000 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 19:15:54.390391 31230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:54.380725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:60781 (local address 127.3.61.129:41967)
0811 19:15:54.380972 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:54.380979 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:54.381242 (+ 263us) server_negotiation.cc:407] Connection header received
0811 19:15:54.381455 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:54.381461 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:54.381567 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:54.381712 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:15:54.382651 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:54.383865 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:54.387261 (+ 3396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:54.388637 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:54.388821 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:54.388902 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:54.389952 (+ 1050us) server_negotiation.cc:299] Negotiation successful
0811 19:15:54.390101 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 19:15:54.390750 31250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:54.380105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.131:37193 (local address 127.3.61.130:43735)
0811 19:15:54.381030 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:54.381058 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 19:15:54.381274 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:15:54.381893 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:15:54.381897 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:15:54.381944 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:15:54.382384 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:54.382482 (+ 98us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:54.384049 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:54.384055 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:54.386934 (+ 2879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:54.386958 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:54.389091 (+ 2133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:54.389114 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:54.389484 (+ 370us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:54.389546 (+ 62us) client_negotiation.cc:769] Sending connection context
0811 19:15:54.389821 (+ 275us) client_negotiation.cc:240] Negotiation successful
0811 19:15:54.390010 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":716,"mutex_wait_us":214,"spinlock_wait_cycles":47872,"thread_start_us":614,"threads_started":1}
I20260811 19:15:54.391202 31249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:54.380199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:41967 (local address 127.3.61.130:60781)
0811 19:15:54.381070 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:54.381093 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 19:15:54.381274 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:15:54.381790 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:15:54.381794 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:15:54.381826 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:15:54.382311 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:54.382328 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:54.384052 (+ 1724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:54.384056 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:54.386934 (+ 2878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:15:54.386958 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:15:54.389139 (+ 2181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:15:54.389149 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:15:54.389479 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:54.389547 (+ 68us) client_negotiation.cc:769] Sending connection context
0811 19:15:54.390625 (+ 1078us) client_negotiation.cc:240] Negotiation successful
0811 19:15:54.390802 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":758,"mutex_wait_us":230,"thread_start_us":842,"threads_started":1}
I20260811 19:15:54.392122 31129 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2540d657eea4171b930aa7bc8e79bcb" candidate_uuid: "4a86881f7b5949c6b4c66b5501d511a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b1650603d4846b0ab3a75db70545cd9" is_pre_election: true
I20260811 19:15:54.392377 30683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2540d657eea4171b930aa7bc8e79bcb" candidate_uuid: "4a86881f7b5949c6b4c66b5501d511a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b8d3747e7a240acb769d8b40c9fff76" is_pre_election: true
I20260811 19:15:54.393144 31129 raft_consensus.cc:2454] T f2540d657eea4171b930aa7bc8e79bcb P 0b1650603d4846b0ab3a75db70545cd9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a86881f7b5949c6b4c66b5501d511a6 in term 0.
I20260811 19:15:54.393244 30683 raft_consensus.cc:2454] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a86881f7b5949c6b4c66b5501d511a6 in term 0.
I20260811 19:15:54.394186 30804 leader_election.cc:302] T f2540d657eea4171b930aa7bc8e79bcb P 4a86881f7b5949c6b4c66b5501d511a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b1650603d4846b0ab3a75db70545cd9, 4a86881f7b5949c6b4c66b5501d511a6; no voters:
I20260811 19:15:54.394831 31244 raft_consensus.cc:2781] T f2540d657eea4171b930aa7bc8e79bcb P 4a86881f7b5949c6b4c66b5501d511a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:15:54.395080 31244 raft_consensus.cc:482] T f2540d657eea4171b930aa7bc8e79bcb P 4a86881f7b5949c6b4c66b5501d511a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:15:54.395188 31244 raft_consensus.cc:3037] T f2540d657eea4171b930aa7bc8e79bcb P 4a86881f7b5949c6b4c66b5501d511a6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:15:54.398510 31244 raft_consensus.cc:504] T f2540d657eea4171b930aa7bc8e79bcb P 4a86881f7b5949c6b4c66b5501d511a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b1650603d4846b0ab3a75db70545cd9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37193 } } peers { permanent_uuid: "4a86881f7b5949c6b4c66b5501d511a6" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41047 } } peers { permanent_uuid: "4b8d3747e7a240acb769d8b40c9fff76" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 41967 } }
I20260811 19:15:54.399340 31244 leader_election.cc:288] T f2540d657eea4171b930aa7bc8e79bcb P 4a86881f7b5949c6b4c66b5501d511a6 [CANDIDATE]: Term 1 election: Requested vote from peers 0b1650603d4846b0ab3a75db70545cd9 (127.3.61.131:37193), 4b8d3747e7a240acb769d8b40c9fff76 (127.3.61.129:41967)
I20260811 19:15:54.399984 31129 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2540d657eea4171b930aa7bc8e79bcb" candidate_uuid: "4a86881f7b5949c6b4c66b5501d511a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b1650603d4846b0ab3a75db70545cd9"
I20260811 19:15:54.399986 30683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2540d657eea4171b930aa7bc8e79bcb" candidate_uuid: "4a86881f7b5949c6b4c66b5501d511a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b8d3747e7a240acb769d8b40c9fff76"
I20260811 19:15:54.400242 31129 raft_consensus.cc:3037] T f2540d657eea4171b930aa7bc8e79bcb P 0b1650603d4846b0ab3a75db70545cd9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:15:54.400242 30683 raft_consensus.cc:3037] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:15:54.403585 31129 raft_consensus.cc:2454] T f2540d657eea4171b930aa7bc8e79bcb P 0b1650603d4846b0ab3a75db70545cd9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a86881f7b5949c6b4c66b5501d511a6 in term 1.
I20260811 19:15:54.403585 30683 raft_consensus.cc:2454] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a86881f7b5949c6b4c66b5501d511a6 in term 1.
I20260811 19:15:54.404103 30804 leader_election.cc:302] T f2540d657eea4171b930aa7bc8e79bcb P 4a86881f7b5949c6b4c66b5501d511a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b1650603d4846b0ab3a75db70545cd9, 4a86881f7b5949c6b4c66b5501d511a6; no voters:
I20260811 19:15:54.404480 31244 raft_consensus.cc:2781] T f2540d657eea4171b930aa7bc8e79bcb P 4a86881f7b5949c6b4c66b5501d511a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:15:54.405083 31244 raft_consensus.cc:686] T f2540d657eea4171b930aa7bc8e79bcb P 4a86881f7b5949c6b4c66b5501d511a6 [term 1 LEADER]: Becoming Leader. State: Replica: 4a86881f7b5949c6b4c66b5501d511a6, State: Running, Role: LEADER
I20260811 19:15:54.405722 31244 consensus_queue.cc:237] T f2540d657eea4171b930aa7bc8e79bcb P 4a86881f7b5949c6b4c66b5501d511a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b1650603d4846b0ab3a75db70545cd9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37193 } } peers { permanent_uuid: "4a86881f7b5949c6b4c66b5501d511a6" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41047 } } peers { permanent_uuid: "4b8d3747e7a240acb769d8b40c9fff76" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 41967 } }
I20260811 19:15:54.411409 30446 catalog_manager.cc:5495] T f2540d657eea4171b930aa7bc8e79bcb P 4a86881f7b5949c6b4c66b5501d511a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4a86881f7b5949c6b4c66b5501d511a6 (127.3.61.130). New cstate: current_term: 1 leader_uuid: "4a86881f7b5949c6b4c66b5501d511a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b1650603d4846b0ab3a75db70545cd9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37193 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a86881f7b5949c6b4c66b5501d511a6" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41047 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b8d3747e7a240acb769d8b40c9fff76" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 41967 } health_report { overall_health: UNKNOWN } } }
I20260811 19:15:54.439234 31232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:54.432656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60280 (local address 127.3.61.130:41047)
0811 19:15:54.432915 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:54.432920 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:54.432940 (+ 20us) server_negotiation.cc:407] Connection header received
0811 19:15:54.433093 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:54.433097 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:54.433190 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:54.433342 (+ 152us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:15:54.433869 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:54.434718 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:54.436907 (+ 2189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:54.437336 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:54.437446 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:54.438061 (+ 615us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:15:54.438607 (+ 546us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:15:54.438721 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:54.438862 (+ 141us) server_negotiation.cc:299] Negotiation successful
0811 19:15:54.438940 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260811 19:15:54.449306 30683 raft_consensus.cc:1261] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76 [term 1 FOLLOWER]: Refusing update from remote peer 4a86881f7b5949c6b4c66b5501d511a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:15:54.449458 31129 raft_consensus.cc:1261] T f2540d657eea4171b930aa7bc8e79bcb P 0b1650603d4846b0ab3a75db70545cd9 [term 1 FOLLOWER]: Refusing update from remote peer 4a86881f7b5949c6b4c66b5501d511a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:15:54.450573 31251 consensus_queue.cc:1035] T f2540d657eea4171b930aa7bc8e79bcb P 4a86881f7b5949c6b4c66b5501d511a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b8d3747e7a240acb769d8b40c9fff76" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 41967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:15:54.451115 31244 consensus_queue.cc:1035] T f2540d657eea4171b930aa7bc8e79bcb P 4a86881f7b5949c6b4c66b5501d511a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b1650603d4846b0ab3a75db70545cd9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:15:54.464998 31256 mvcc.cc:205] Tried to move back new op lower bound from 7317404690213818368 to 7317404690050854912. Current Snapshot: MvccSnapshot[applied={T|T < 7317404690213818368}]
I20260811 19:15:54.480830 30446 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56120:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:15:54.481195 30446 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56120:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:15:54.485894 30446 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0: Sending DeleteTablet for 3 replicas of tablet f2540d657eea4171b930aa7bc8e79bcb
I20260811 19:15:54.487668 31091 tablet_service.cc:1514] Processing DeleteTablet for tablet f2540d657eea4171b930aa7bc8e79bcb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:15:54 UTC) from {username='kudu', principal='kudu/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:33248
I20260811 19:15:54.487988 30868 tablet_service.cc:1514] Processing DeleteTablet for tablet f2540d657eea4171b930aa7bc8e79bcb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:15:54 UTC) from {username='kudu', principal='kudu/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:60272
I20260811 19:15:54.487982 30656 tablet_service.cc:1514] Processing DeleteTablet for tablet f2540d657eea4171b930aa7bc8e79bcb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:15:54 UTC) from {username='kudu', principal='kudu/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:55090
I20260811 19:15:54.488969 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 30511
I20260811 19:15:54.501096 31274 tablet_replica.cc:330] T f2540d657eea4171b930aa7bc8e79bcb P 4a86881f7b5949c6b4c66b5501d511a6: stopping tablet replica
I20260811 19:15:54.501739 31274 raft_consensus.cc:2229] T f2540d657eea4171b930aa7bc8e79bcb P 4a86881f7b5949c6b4c66b5501d511a6 [term 1 LEADER]: Raft consensus shutting down.
I20260811 19:15:54.502775 31274 raft_consensus.cc:2258] T f2540d657eea4171b930aa7bc8e79bcb P 4a86881f7b5949c6b4c66b5501d511a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:15:54.506346 31274 ts_tablet_manager.cc:1905] T f2540d657eea4171b930aa7bc8e79bcb P 4a86881f7b5949c6b4c66b5501d511a6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:15:54.516287 31274 ts_tablet_manager.cc:1918] T f2540d657eea4171b930aa7bc8e79bcb P 4a86881f7b5949c6b4c66b5501d511a6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:15:54.516870 31274 log.cc:1196] T f2540d657eea4171b930aa7bc8e79bcb P 4a86881f7b5949c6b4c66b5501d511a6: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-1/wal/wals/f2540d657eea4171b930aa7bc8e79bcb
I20260811 19:15:54.518090 31274 ts_tablet_manager.cc:1939] T f2540d657eea4171b930aa7bc8e79bcb P 4a86881f7b5949c6b4c66b5501d511a6: Deleting consensus metadata
I20260811 19:15:54.520553 30428 catalog_manager.cc:4841] TS 4a86881f7b5949c6b4c66b5501d511a6 (127.3.61.130:41047): tablet f2540d657eea4171b930aa7bc8e79bcb (table test-table [id=2ed3889ac2364292bd89a70eca0d70b5]) successfully deleted
I20260811 19:15:54.530361 31271 tablet_replica.cc:330] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76: stopping tablet replica
I20260811 19:15:54.530912 31230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:54.490155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55104 (local address 127.3.61.129:41967)
0811 19:15:54.490337 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:54.490344 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:54.490364 (+ 20us) server_negotiation.cc:407] Connection header received
0811 19:15:54.490432 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:54.490438 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:54.490531 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:54.490640 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:15:54.492648 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:54.493879 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:54.496337 (+ 2458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:54.497034 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:54.497298 (+ 264us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:54.518492 (+ 21194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:15:54.518519 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:15:54.518543 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:15:54.518604 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:15:54.523887 (+ 5283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:54.525029 (+ 1142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:54.525040 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:54.525044 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:54.525231 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:54.526687 (+ 1456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:54.526694 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:54.526698 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:54.527866 (+ 1168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:15:54.528195 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:54.530173 (+ 1978us) server_negotiation.cc:299] Negotiation successful
0811 19:15:54.530484 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 19:15:54.531733 31271 raft_consensus.cc:2229] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:15:54.532199 30757 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:15:54.534261 31270 tablet_replica.cc:330] T f2540d657eea4171b930aa7bc8e79bcb P 0b1650603d4846b0ab3a75db70545cd9: stopping tablet replica
I20260811 19:15:54.534693 31270 raft_consensus.cc:2229] T f2540d657eea4171b930aa7bc8e79bcb P 0b1650603d4846b0ab3a75db70545cd9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:15:54.535228 31270 raft_consensus.cc:2258] T f2540d657eea4171b930aa7bc8e79bcb P 0b1650603d4846b0ab3a75db70545cd9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:15:54.542408 31270 ts_tablet_manager.cc:1905] T f2540d657eea4171b930aa7bc8e79bcb P 0b1650603d4846b0ab3a75db70545cd9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:15:54.550971 31270 ts_tablet_manager.cc:1918] T f2540d657eea4171b930aa7bc8e79bcb P 0b1650603d4846b0ab3a75db70545cd9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:15:54.551406 31270 log.cc:1196] T f2540d657eea4171b930aa7bc8e79bcb P 0b1650603d4846b0ab3a75db70545cd9: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-2/wal/wals/f2540d657eea4171b930aa7bc8e79bcb
I20260811 19:15:54.552167 31270 ts_tablet_manager.cc:1939] T f2540d657eea4171b930aa7bc8e79bcb P 0b1650603d4846b0ab3a75db70545cd9: Deleting consensus metadata
I20260811 19:15:54.554492 30428 catalog_manager.cc:4841] TS 0b1650603d4846b0ab3a75db70545cd9 (127.3.61.131:37193): tablet f2540d657eea4171b930aa7bc8e79bcb (table test-table [id=2ed3889ac2364292bd89a70eca0d70b5]) successfully deleted
I20260811 19:15:54.642871 31271 raft_consensus.cc:2258] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:15:54.644640 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 30511
I20260811 19:15:54.681610 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 30773
W20260811 19:15:54.681728 30424 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.61.129:41967 (error 108)
I20260811 19:15:54.684182 31281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:54.683141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:41967 (local address 127.0.0.1:55114)
0811 19:15:54.683817 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:54.683968 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.129:41967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":276,"threads_started":1}
W20260811 19:15:54.684517 30424 catalog_manager.cc:4584] TS 4b8d3747e7a240acb769d8b40c9fff76 (127.3.61.129:41967): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f2540d657eea4171b930aa7bc8e79bcb: Network error: Client connection negotiation failed: client connection to 127.3.61.129:41967: connect: Connection refused (error 111)
I20260811 19:15:54.699605 31232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:54.682831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60284 (local address 127.3.61.130:41047)
0811 19:15:54.683024 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:54.683031 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:54.683055 (+ 24us) server_negotiation.cc:407] Connection header received
0811 19:15:54.683314 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:54.683321 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:54.683429 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:54.683561 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:15:54.685285 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:54.686371 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:54.687863 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:54.688289 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:54.688410 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:54.691912 (+ 3502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:15:54.691932 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:15:54.691955 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:15:54.692042 (+ 87us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:15:54.695876 (+ 3834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:54.696885 (+ 1009us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:54.696896 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:54.696898 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:54.697105 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:54.697534 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:54.697539 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:54.697541 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:54.698078 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:15:54.698452 (+ 374us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:54.698925 (+ 473us) server_negotiation.cc:299] Negotiation successful
0811 19:15:54.699191 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 19:15:54.700536 30967 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:15:54.738919 31281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:54.738080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:41967 (local address 127.0.0.1:55126)
0811 19:15:54.738492 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:54.738702 (+ 210us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.129:41967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
I20260811 19:15:54.740347 31281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:54.739917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:41967 (local address 127.0.0.1:55138)
0811 19:15:54.740073 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:54.740164 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.129:41967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 19:15:54.801368 31281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:54.800815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:41967 (local address 127.0.0.1:55146)
0811 19:15:54.801075 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:54.801180 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.129:41967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260811 19:15:54.802891 31281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:54.802441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:41967 (local address 127.0.0.1:55150)
0811 19:15:54.802612 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:54.802709 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.129:41967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 19:15:54.824105 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 30773
I20260811 19:15:54.848793 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 30981
I20260811 19:15:54.866273 31225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:54.849879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33252 (local address 127.3.61.131:37193)
0811 19:15:54.850075 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 19:15:54.850080 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:15:54.850100 (+ 20us) server_negotiation.cc:407] Connection header received
0811 19:15:54.850338 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:15:54.850344 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:15:54.850423 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:15:54.850549 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:15:54.852244 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:54.853132 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:54.854603 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:15:54.855206 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:15:54.855323 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:15:54.858804 (+ 3481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:15:54.858821 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:15:54.858840 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:15:54.858887 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:15:54.862398 (+ 3511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:54.863546 (+ 1148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:54.863559 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:54.863561 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:54.863791 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:15:54.864269 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:15:54.864273 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:15:54.864276 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:15:54.864744 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:15:54.865111 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0811 19:15:54.865445 (+ 334us) server_negotiation.cc:299] Negotiation successful
0811 19:15:54.865877 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 19:15:54.867022 31202 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:15:54.884054 31281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:54.883398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:41967 (local address 127.0.0.1:55158)
0811 19:15:54.883670 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:54.883775 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.129:41967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
I20260811 19:15:54.885411 31281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:54.884932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:41967 (local address 127.0.0.1:55160)
0811 19:15:54.885152 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:54.885240 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.129:41967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260811 19:15:54.965268 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 30981
I20260811 19:15:54.992250 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 30401
I20260811 19:15:54.993777 30502 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:15:55.124047 31281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:15:55.123361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:41967 (local address 127.0.0.1:55166)
0811 19:15:55.123725 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0811 19:15:55.123854 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.129:41967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260811 19:15:55.124115 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 30401
I20260811 19:16:15.155800 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:43283
--webserver_interface=127.3.61.190
--webserver_port=42311
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:35451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.61.190:43283
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:16:15.566967 31285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:16:15.567327 31285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:16:15.567435 31285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:16:15.577216 31285 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 19:16:15.577389 31285 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 19:16:15.577430 31285 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 19:16:15.577494 31285 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:16:15.577556 31285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:16:15.577629 31285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:16:15.577714 31285 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:16:15.577786 31285 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:16:15.590279 31285 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:35451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.3.61.190:43283
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:43283
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.61.190
--webserver_port=42311
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:16:15.591682 31285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:16:15.593122 31285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:16:15.606204 31295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:16:15.606428 31299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:16:15.607172 31301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:16:15.607931 31285 server_base.cc:1034] running on GCE node
I20260811 19:16:15.608875 31285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:16:15.610605 31285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:16:15.611874 31285 hybrid_clock.cc:648] HybridClock initialized: now 1786475775611798 us; error 49 us; skew 500 ppm
I20260811 19:16:15.612334 31285 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:16:15 dist-test-slave-t7g5 krb5kdc[29479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475775, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:16:15.622439 31285 init.cc:377] Logged in from keytab as kudu/127.3.61.190@KRBTEST.COM (short username kudu)
I20260811 19:16:15.624488 31285 webserver.cc:460] Webserver started at http://127.3.61.190:42311/ using document root <none> and password file <none>
I20260811 19:16:15.625720 31285 fs_manager.cc:359] Metadata directory not provided
I20260811 19:16:15.625871 31285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:16:15.633064 31285 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.006s
I20260811 19:16:15.636714 31315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:15.638675 31285 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.002s
I20260811 19:16:15.638888 31285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/master-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/master-0/wal
uuid: "41c09bf6448a4760973e8a7897b4ccd0"
format_stamp: "Formatted at 2026-08-11 19:15:28 on dist-test-slave-t7g5"
server_key: "3d241892a73f429e804e9ec3331fb46e"
server_key_iv: "a47e4b8a88ff5ed8f5c0e2cf7e78072e"
server_key_version: "encryptionkey@0"
I20260811 19:16:15.640014 31285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:16:15.672302 31285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:16:15.678227 31285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:16:15.678578 31285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:16:15.701442 31285 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.190:43283
I20260811 19:16:15.701448 31407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.190:43283 every 8 connection(s)
I20260811 19:16:15.703282 31285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
I20260811 19:16:15.710759 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 31285
I20260811 19:16:15.711853 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.129:41967
--local_ip_for_outbound_sockets=127.3.61.129
--tserver_master_addrs=127.3.61.190:43283
--webserver_port=44995
--webserver_interface=127.3.61.129
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab
--principal=kudu/127.3.61.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:35451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:16:15.713567 31408 sys_catalog.cc:263] Verifying existing consensus state
I20260811 19:16:15.718516 31408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0: Bootstrap starting.
I20260811 19:16:15.744756 31408 log.cc:824] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0: Log is configured to *not* fsync() on all Append() calls
I20260811 19:16:15.766774 31408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 19:16:15.767416 31408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0: Bootstrap complete.
I20260811 19:16:15.777249 31408 raft_consensus.cc:348] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41c09bf6448a4760973e8a7897b4ccd0" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 43283 } }
I20260811 19:16:15.778818 31408 raft_consensus.cc:729] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41c09bf6448a4760973e8a7897b4ccd0, State: Initialized, Role: FOLLOWER
I20260811 19:16:15.779860 31408 consensus_queue.cc:260] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41c09bf6448a4760973e8a7897b4ccd0" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 43283 } }
I20260811 19:16:15.780210 31408 raft_consensus.cc:388] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:16:15.780359 31408 raft_consensus.cc:482] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:16:15.780541 31408 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0 [term 2 FOLLOWER]: Advancing to term 3
I20260811 19:16:15.785166 31408 raft_consensus.cc:504] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41c09bf6448a4760973e8a7897b4ccd0" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 43283 } }
I20260811 19:16:15.785843 31408 leader_election.cc:302] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 41c09bf6448a4760973e8a7897b4ccd0; no voters:
I20260811 19:16:15.786556 31408 leader_election.cc:288] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 19:16:15.786736 31416 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 19:16:15.787856 31416 raft_consensus.cc:686] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0 [term 3 LEADER]: Becoming Leader. State: Replica: 41c09bf6448a4760973e8a7897b4ccd0, State: Running, Role: LEADER
I20260811 19:16:15.788576 31416 consensus_queue.cc:237] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41c09bf6448a4760973e8a7897b4ccd0" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 43283 } }
I20260811 19:16:15.788918 31408 sys_catalog.cc:564] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:16:15.791263 31419 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 41c09bf6448a4760973e8a7897b4ccd0. Latest consensus state: current_term: 3 leader_uuid: "41c09bf6448a4760973e8a7897b4ccd0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41c09bf6448a4760973e8a7897b4ccd0" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 43283 } } }
I20260811 19:16:15.791697 31419 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0 [sys.catalog]: This master's current role is: LEADER
I20260811 19:16:15.792430 31418 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "41c09bf6448a4760973e8a7897b4ccd0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41c09bf6448a4760973e8a7897b4ccd0" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 43283 } } }
I20260811 19:16:15.792750 31418 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0 [sys.catalog]: This master's current role is: LEADER
I20260811 19:16:15.796465 31424 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:16:15.808364 31424 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:16:15.810359 31424 catalog_manager.cc:1259] Loaded cluster ID: 3f91653b5f7b4343ad108648ffeec1e3
I20260811 19:16:15.810596 31424 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:16:15.815164 31438 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 19:16:15.815897 31424 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:16:15.817492 31424 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0: Loaded TSK: 0
I20260811 19:16:15.817618 31424 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 41c09bf6448a4760973e8a7897b4ccd0: Loaded TSK: 1
F20260811 19:16:15.818523 31424 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fb5f28069ad google::LogMessage::Fail() at ??:0
@ 0x7fb5f280988a google::LogMessage::SendToLog() at ??:0
@ 0x7fb5f28064de google::LogMessage::Flush() at ??:0
@ 0x7fb5f2807a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fb5f656a7d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fb5f655973e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fb6082351c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fb608234783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fb6082c02a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7fb6082c00c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7fb60830a5e9 std::function<>::operator()() at ??:0
@ 0x7fb608238cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fb608236c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fb6082be610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fb6082be439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fb5f5452345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fb5f545591c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fb5f5455799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fb5f5430eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fb5f870a6db start_thread at ??:0
@ 0x7fb5f16a061f clone at ??:0
W20260811 19:16:16.147477 31413 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:16:16.147840 31413 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:16:16.147979 31413 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:16:16.156080 31413 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:16:16.156287 31413 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:16:16.156467 31413 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.129
I20260811 19:16:16.167229 31413 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:35451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab
--principal=kudu/127.3.61.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.129:41967
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.61.129
--webserver_port=44995
--tserver_master_addrs=127.3.61.190:43283
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.129
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:16:16.168725 31413 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:16:16.170399 31413 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:16:16.186213 31489 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:16:16.186311 31491 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:16:16.186260 31488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:16:16.187588 31413 server_base.cc:1034] running on GCE node
I20260811 19:16:16.188342 31413 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:16:16.189582 31413 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:16:16.190896 31413 hybrid_clock.cc:648] HybridClock initialized: now 1786475776190806 us; error 63 us; skew 500 ppm
I20260811 19:16:16.191274 31413 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:16:16 dist-test-slave-t7g5 krb5kdc[29479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475776, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:16:16.196066 31413 init.cc:377] Logged in from keytab as kudu/127.3.61.129@KRBTEST.COM (short username kudu)
I20260811 19:16:16.198460 31413 webserver.cc:460] Webserver started at http://127.3.61.129:44995/ using document root <none> and password file <none>
I20260811 19:16:16.199296 31413 fs_manager.cc:359] Metadata directory not provided
I20260811 19:16:16.199441 31413 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:16:16.206135 31413 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 19:16:16.210480 31504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:16.212069 31413 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 19:16:16.212272 31413 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-0/wal
uuid: "4b8d3747e7a240acb769d8b40c9fff76"
format_stamp: "Formatted at 2026-08-11 19:15:29 on dist-test-slave-t7g5"
server_key: "e7f851f8670b06e7bf3d0cd48f0abe3a"
server_key_iv: "f9cbae46258a0c9ac2c14426e0da52d4"
server_key_version: "encryptionkey@0"
I20260811 19:16:16.213333 31413 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:16:16.231736 31413 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:16:16.237713 31413 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:16:16.238050 31413 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:16:16.239387 31413 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:16:16.242066 31516 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260811 19:16:16.250286 31413 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 19:16:16.250499 31413 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.009s user 0.001s sys 0.000s
I20260811 19:16:16.250645 31413 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 19:16:16.254374 31413 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 19:16:16.254480 31413 ts_tablet_manager.cc:589] Time spent register tablets: real 0.004s user 0.003s sys 0.000s
I20260811 19:16:16.254997 31516 tablet_bootstrap.cc:492] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76: Bootstrap starting.
I20260811 19:16:16.266731 31516 log.cc:824] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76: Log is configured to *not* fsync() on all Append() calls
I20260811 19:16:16.284585 31516 tablet_bootstrap.cc:492] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 19:16:16.285633 31516 tablet_bootstrap.cc:492] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76: Bootstrap complete.
I20260811 19:16:16.286316 31516 ts_tablet_manager.cc:1397] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76: Time spent bootstrapping tablet: real 0.032s user 0.022s sys 0.008s
I20260811 19:16:16.296630 31516 raft_consensus.cc:348] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b1650603d4846b0ab3a75db70545cd9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37193 } } peers { permanent_uuid: "4a86881f7b5949c6b4c66b5501d511a6" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41047 } } peers { permanent_uuid: "4b8d3747e7a240acb769d8b40c9fff76" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 41967 } }
I20260811 19:16:16.297787 31516 raft_consensus.cc:729] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b8d3747e7a240acb769d8b40c9fff76, State: Initialized, Role: FOLLOWER
I20260811 19:16:16.298813 31516 consensus_queue.cc:260] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b1650603d4846b0ab3a75db70545cd9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37193 } } peers { permanent_uuid: "4a86881f7b5949c6b4c66b5501d511a6" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41047 } } peers { permanent_uuid: "4b8d3747e7a240acb769d8b40c9fff76" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 41967 } }
I20260811 19:16:16.301604 31516 ts_tablet_manager.cc:1428] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76: Time spent starting tablet: real 0.015s user 0.009s sys 0.004s
I20260811 19:16:16.301749 31413 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.129:41967
I20260811 19:16:16.301646 31707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.129:41967 every 8 connection(s)
I20260811 19:16:16.303259 31413 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
I20260811 19:16:16.306573 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 31413
I20260811 19:16:16.307417 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.130:41047
--local_ip_for_outbound_sockets=127.3.61.130
--tserver_master_addrs=127.3.61.190:43283
--webserver_port=43429
--webserver_interface=127.3.61.130
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab
--principal=kudu/127.3.61.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:35451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:16:16.309458 31711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:16.306865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43283 (local address 127.3.61.129:39283)
0811 19:16:16.307877 (+ 1012us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:16.308123 (+ 246us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:43283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":674,"thread_start_us":397,"threads_started":1}
W20260811 19:16:16.310267 31708 heartbeater.cc:644] Failed to heartbeat to 127.3.61.190:43283 (0 consecutive failures): Network error: Failed to ping master at 127.3.61.190:43283: Client connection negotiation failed: client connection to 127.3.61.190:43283: connect: Connection refused (error 111)
I20260811 19:16:16.311198 31711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:16.310834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43283 (local address 127.3.61.129:49161)
0811 19:16:16.311005 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:16.311085 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:43283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 19:16:16.312158 31711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:16.311770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43283 (local address 127.3.61.129:56645)
0811 19:16:16.311940 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:16.312020 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:43283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260811 19:16:16.312386 31708 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 19:16:16.690119 31712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:16:16.690446 31712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:16:16.690572 31712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:16:16.698441 31712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:16:16.698587 31712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:16:16.698737 31712 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.130
I20260811 19:16:16.709874 31712 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:35451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab
--principal=kudu/127.3.61.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.130:41047
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.61.130
--webserver_port=43429
--tserver_master_addrs=127.3.61.190:43283
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.130
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:16:16.711334 31712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:16:16.712715 31712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:16:16.729537 31721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:16:16.729508 31717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:16:16.729584 31719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:16:16.730419 31712 server_base.cc:1034] running on GCE node
I20260811 19:16:16.731101 31712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:16:16.732173 31712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:16:16.733408 31712 hybrid_clock.cc:648] HybridClock initialized: now 1786475776733363 us; error 34 us; skew 500 ppm
I20260811 19:16:16.733810 31712 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:16:16 dist-test-slave-t7g5 krb5kdc[29479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475776, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:16:16.737267 31712 init.cc:377] Logged in from keytab as kudu/127.3.61.130@KRBTEST.COM (short username kudu)
I20260811 19:16:16.739464 31712 webserver.cc:460] Webserver started at http://127.3.61.130:43429/ using document root <none> and password file <none>
I20260811 19:16:16.740232 31712 fs_manager.cc:359] Metadata directory not provided
I20260811 19:16:16.740367 31712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:16:16.746861 31712 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 19:16:16.750226 31730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:16.752199 31712 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260811 19:16:16.752391 31712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-1/wal
uuid: "4a86881f7b5949c6b4c66b5501d511a6"
format_stamp: "Formatted at 2026-08-11 19:15:30 on dist-test-slave-t7g5"
server_key: "280d131b916c7795b0661227a79df126"
server_key_iv: "8cd44d4237f74d5e912f97328baed288"
server_key_version: "encryptionkey@0"
I20260811 19:16:16.753460 31712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:16:16.794658 31712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:16:16.799831 31712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:16:16.800254 31712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:16:16.801504 31712 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:16:16.803243 31712 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:16:16.803370 31712 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:16.803500 31712 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:16:16.803587 31712 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:16.841861 31712 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.130:41047
I20260811 19:16:16.842028 31915 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.130:41047 every 8 connection(s)
I20260811 19:16:16.843487 31712 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
I20260811 19:16:16.848003 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 31712
I20260811 19:16:16.848548 31918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:16.846590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43283 (local address 127.3.61.130:51743)
0811 19:16:16.847373 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:16.847609 (+ 236us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:43283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":291,"threads_started":1}
I20260811 19:16:16.848778 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.131:37193
--local_ip_for_outbound_sockets=127.3.61.131
--tserver_master_addrs=127.3.61.190:43283
--webserver_port=36513
--webserver_interface=127.3.61.131
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab
--principal=kudu/127.3.61.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:35451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:16:16.849298 31916 heartbeater.cc:644] Failed to heartbeat to 127.3.61.190:43283 (0 consecutive failures): Network error: Failed to ping master at 127.3.61.190:43283: Client connection negotiation failed: client connection to 127.3.61.190:43283: connect: Connection refused (error 111)
I20260811 19:16:16.850354 31918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:16.849861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43283 (local address 127.3.61.130:33613)
0811 19:16:16.850020 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:16.850218 (+ 198us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:43283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 19:16:16.851235 31918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:16.850896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43283 (local address 127.3.61.130:53443)
0811 19:16:16.851040 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:16.851121 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:43283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 19:16:16.851477 31916 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 19:16:17.241360 31920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:16:17.242049 31920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:16:17.242286 31920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:16:17.251031 31920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:16:17.251186 31920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:16:17.251356 31920 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.131
I20260811 19:16:17.263182 31920 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:35451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab
--principal=kudu/127.3.61.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.131:37193
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.61.131
--webserver_port=36513
--tserver_master_addrs=127.3.61.190:43283
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.131
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:16:17.264671 31920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:16:17.266065 31920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:16:17.279963 31928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:16:17.279935 31929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:16:17.280119 31933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:16:17.281860 31920 server_base.cc:1034] running on GCE node
I20260811 19:16:17.282563 31920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:16:17.283648 31920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:16:17.284884 31920 hybrid_clock.cc:648] HybridClock initialized: now 1786475777284815 us; error 53 us; skew 500 ppm
I20260811 19:16:17.285255 31920 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:16:17 dist-test-slave-t7g5 krb5kdc[29479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475777, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:16:17.288595 31920 init.cc:377] Logged in from keytab as kudu/127.3.61.131@KRBTEST.COM (short username kudu)
I20260811 19:16:17.290410 31920 webserver.cc:460] Webserver started at http://127.3.61.131:36513/ using document root <none> and password file <none>
I20260811 19:16:17.291157 31920 fs_manager.cc:359] Metadata directory not provided
I20260811 19:16:17.291273 31920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:16:17.297081 31920 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 19:16:17.299975 31947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:17.302042 31920 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 19:16:17.302251 31920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-2/wal
uuid: "0b1650603d4846b0ab3a75db70545cd9"
format_stamp: "Formatted at 2026-08-11 19:15:30 on dist-test-slave-t7g5"
server_key: "57563b6588e292f0c6dac01ca95bbbae"
server_key_iv: "9303a5f0bde5aa3283ddcc2dad2497a8"
server_key_version: "encryptionkey@0"
I20260811 19:16:17.303578 31920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:16:17.314167 31957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:17.313118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43283 (local address 127.3.61.129:56769)
0811 19:16:17.313848 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:17.313959 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:43283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":411,"threads_started":1}
I20260811 19:16:17.323346 31920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:16:17.328148 31920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:16:17.328467 31920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:16:17.329795 31920 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:16:17.331231 31920 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:16:17.331339 31920 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:17.331445 31920 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:16:17.331528 31920 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:17.367566 31920 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.131:37193
I20260811 19:16:17.367626 32141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.131:37193 every 8 connection(s)
I20260811 19:16:17.369059 31920 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
I20260811 19:16:17.374770 32145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:17.372401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43283 (local address 127.3.61.131:40761)
0811 19:16:17.373459 (+ 1058us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:17.373834 (+ 375us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:43283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":653,"thread_start_us":392,"threads_started":1}
W20260811 19:16:17.375562 32143 heartbeater.cc:644] Failed to heartbeat to 127.3.61.190:43283 (0 consecutive failures): Network error: Failed to ping master at 127.3.61.190:43283: Client connection negotiation failed: client connection to 127.3.61.190:43283: connect: Connection refused (error 111)
I20260811 19:16:17.376423 32145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:17.376066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43283 (local address 127.3.61.131:41203)
0811 19:16:17.376240 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:17.376308 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:43283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 19:16:17.377153 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 31920
I20260811 19:16:17.377275 32145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:17.376954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43283 (local address 127.3.61.131:50435)
0811 19:16:17.377090 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:17.377166 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:43283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 19:16:17.377697 32143 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 19:16:17.795704 32150 raft_consensus.cc:482] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:16:17.796075 32150 raft_consensus.cc:504] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b1650603d4846b0ab3a75db70545cd9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37193 } } peers { permanent_uuid: "4a86881f7b5949c6b4c66b5501d511a6" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41047 } } peers { permanent_uuid: "4b8d3747e7a240acb769d8b40c9fff76" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 41967 } }
I20260811 19:16:17.797946 32150 leader_election.cc:288] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0b1650603d4846b0ab3a75db70545cd9 (127.3.61.131:37193), 4a86881f7b5949c6b4c66b5501d511a6 (127.3.61.130:41047)
Aug 11 19:16:17 dist-test-slave-t7g5 krb5kdc[29479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475776, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.129@KRBTEST.COM for kudu/127.3.61.130@KRBTEST.COM
Aug 11 19:16:17 dist-test-slave-t7g5 krb5kdc[29479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475776, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.129@KRBTEST.COM for kudu/127.3.61.131@KRBTEST.COM
I20260811 19:16:17.825250 31957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:17.798438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.131:37193 (local address 127.3.61.129:44753)
0811 19:16:17.798715 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:17.798776 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 19:16:17.799737 (+ 961us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:16:17.803128 (+ 3391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:16:17.803139 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:16:17.803949 (+ 810us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:16:17.804914 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:17.804940 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:17.806740 (+ 1800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:17.806744 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:17.808123 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:17.808138 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:17.809498 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:17.809503 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:17.810033 (+ 530us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:17.811182 (+ 1149us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:16:17.811228 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:16:17.814286 (+ 3058us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:16:17.819310 (+ 5024us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:16:17.819324 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:16:17.819351 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:16:17.820437 (+ 1086us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:16:17.821392 (+ 955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:16:17.821398 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:16:17.821403 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:16:17.821790 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:16:17.823174 (+ 1384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:16:17.823183 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:16:17.823829 (+ 646us) client_negotiation.cc:769] Sending connection context
0811 19:16:17.824123 (+ 294us) client_negotiation.cc:240] Negotiation successful
0811 19:16:17.824794 (+ 671us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"mutex_wait_us":156,"spinlock_wait_cycles":79104}
I20260811 19:16:17.825353 32151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:17.798746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:44753 (local address 127.3.61.131:37193)
0811 19:16:17.799618 (+ 872us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:17.799637 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:17.799696 (+ 59us) server_negotiation.cc:407] Connection header received
0811 19:16:17.801842 (+ 2146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:17.801852 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:17.802507 (+ 655us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:17.802980 (+ 473us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:16:17.805164 (+ 2184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:17.806544 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:17.808273 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:17.809253 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:17.809492 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:17.814585 (+ 5093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:16:17.814594 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:16:17.814608 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:16:17.814663 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:16:17.818955 (+ 4292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:16:17.820727 (+ 1772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:16:17.820741 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:16:17.820751 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:16:17.821196 (+ 445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:16:17.821946 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:16:17.821951 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:16:17.821954 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:16:17.822974 (+ 1020us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:16:17.823320 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:17.824517 (+ 1197us) server_negotiation.cc:299] Negotiation successful
0811 19:16:17.824935 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":675,"thread_start_us":487,"threads_started":1}
I20260811 19:16:17.825935 32153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:17.800161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:57805 (local address 127.3.61.130:41047)
0811 19:16:17.800691 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:17.800701 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:17.800730 (+ 29us) server_negotiation.cc:407] Connection header received
0811 19:16:17.802239 (+ 1509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:17.802247 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:17.802631 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:17.802949 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:16:17.805377 (+ 2428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:17.806928 (+ 1551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:17.808487 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:17.809492 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:17.809701 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:17.814472 (+ 4771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:16:17.814487 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:16:17.814507 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:16:17.814572 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:16:17.818999 (+ 4427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:16:17.820726 (+ 1727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:16:17.820741 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:16:17.820751 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:16:17.821196 (+ 445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:16:17.823574 (+ 2378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:16:17.823581 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:16:17.823584 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:16:17.824165 (+ 581us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:16:17.824362 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:17.825144 (+ 782us) server_negotiation.cc:299] Negotiation successful
0811 19:16:17.825586 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":272,"threads_started":1}
I20260811 19:16:17.826141 32152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:17.798949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.130:41047 (local address 127.3.61.129:57805)
0811 19:16:17.799987 (+ 1038us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:17.800011 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 19:16:17.800187 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:16:17.803128 (+ 2941us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:16:17.803139 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:16:17.803949 (+ 810us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:16:17.804915 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:17.805030 (+ 115us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:17.807118 (+ 2088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:17.807122 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:17.808319 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:17.808334 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:17.809818 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:17.809828 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:17.810121 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:17.811182 (+ 1061us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:16:17.811225 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:16:17.814088 (+ 2863us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:16:17.819310 (+ 5222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:16:17.819324 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:16:17.819350 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:16:17.820437 (+ 1087us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:16:17.821392 (+ 955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:16:17.821401 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:16:17.821405 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:16:17.821790 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:16:17.824341 (+ 2551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:16:17.824346 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:16:17.824640 (+ 294us) client_negotiation.cc:769] Sending connection context
0811 19:16:17.824844 (+ 204us) client_negotiation.cc:240] Negotiation successful
0811 19:16:17.825124 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":844,"mutex_wait_us":56,"spinlock_wait_cycles":68352,"thread_start_us":707,"threads_started":1}
W20260811 19:16:17.829085 31508 leader_election.cc:341] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0b1650603d4846b0ab3a75db70545cd9 (127.3.61.131:37193): Not found: Tablet not found: f2540d657eea4171b930aa7bc8e79bcb
W20260811 19:16:17.829447 31508 leader_election.cc:341] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a86881f7b5949c6b4c66b5501d511a6 (127.3.61.130:41047): Not found: Tablet not found: f2540d657eea4171b930aa7bc8e79bcb
I20260811 19:16:17.829567 31508 leader_election.cc:302] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b8d3747e7a240acb769d8b40c9fff76; no voters: 0b1650603d4846b0ab3a75db70545cd9, 4a86881f7b5949c6b4c66b5501d511a6
I20260811 19:16:17.830016 32150 raft_consensus.cc:2726] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 19:16:17.853372 32154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:17.852294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43283 (local address 127.3.61.130:47247)
0811 19:16:17.852944 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:17.853103 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:43283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":365,"threads_started":1}
I20260811 19:16:18.315570 31957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:18.315104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43283 (local address 127.3.61.129:35589)
0811 19:16:18.315282 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:18.315376 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:43283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 19:16:18.382678 32158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:18.381146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43283 (local address 127.3.61.131:34221)
0811 19:16:18.382237 (+ 1091us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:18.382382 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:43283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":926,"thread_start_us":788,"threads_started":1}
I20260811 19:16:18.855499 32159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:18.854642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43283 (local address 127.3.61.130:52105)
0811 19:16:18.855218 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:18.855318 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:43283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":337,"threads_started":1}
I20260811 19:16:19.318419 32162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:19.317049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43283 (local address 127.3.61.129:40627)
0811 19:16:19.318136 (+ 1087us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:19.318236 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:43283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":955,"thread_start_us":827,"threads_started":1}
I20260811 19:16:19.385108 32163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:19.384184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43283 (local address 127.3.61.131:38477)
0811 19:16:19.384767 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:19.384882 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:43283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":306,"threads_started":1}
I20260811 19:16:19.803473 32164 raft_consensus.cc:482] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:16:19.803714 32164 raft_consensus.cc:504] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b1650603d4846b0ab3a75db70545cd9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37193 } } peers { permanent_uuid: "4a86881f7b5949c6b4c66b5501d511a6" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41047 } } peers { permanent_uuid: "4b8d3747e7a240acb769d8b40c9fff76" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 41967 } }
I20260811 19:16:19.804399 32164 leader_election.cc:288] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0b1650603d4846b0ab3a75db70545cd9 (127.3.61.131:37193), 4a86881f7b5949c6b4c66b5501d511a6 (127.3.61.130:41047)
W20260811 19:16:19.805892 31508 leader_election.cc:341] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a86881f7b5949c6b4c66b5501d511a6 (127.3.61.130:41047): Not found: Tablet not found: f2540d657eea4171b930aa7bc8e79bcb
W20260811 19:16:19.806138 31508 leader_election.cc:341] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0b1650603d4846b0ab3a75db70545cd9 (127.3.61.131:37193): Not found: Tablet not found: f2540d657eea4171b930aa7bc8e79bcb
I20260811 19:16:19.806257 31508 leader_election.cc:302] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b8d3747e7a240acb769d8b40c9fff76; no voters: 0b1650603d4846b0ab3a75db70545cd9, 4a86881f7b5949c6b4c66b5501d511a6
I20260811 19:16:19.806557 32164 raft_consensus.cc:2726] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 19:16:19.857923 32165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:19.856629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43283 (local address 127.3.61.130:36713)
0811 19:16:19.857411 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:19.857627 (+ 216us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:43283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":604,"thread_start_us":408,"threads_started":1}
I20260811 19:16:20.320698 32166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:20.319835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43283 (local address 127.3.61.129:40485)
0811 19:16:20.320405 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:20.320522 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:43283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":297,"threads_started":1}
I20260811 19:16:20.387385 32168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:20.386504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43283 (local address 127.3.61.131:55085)
0811 19:16:20.387085 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:20.387193 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:43283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":308,"threads_started":1}
I20260811 19:16:20.860146 32169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:20.859009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43283 (local address 127.3.61.130:38905)
0811 19:16:20.859690 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:20.859811 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:43283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":329,"threads_started":1}
I20260811 19:16:21.323642 32170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:21.322093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43283 (local address 127.3.61.129:42407)
0811 19:16:21.322929 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:21.323296 (+ 367us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:43283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":613,"thread_start_us":450,"threads_started":1}
I20260811 19:16:21.389786 32173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:21.388830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43283 (local address 127.3.61.131:35997)
0811 19:16:21.389410 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:21.389516 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:43283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":302,"threads_started":1}
I20260811 19:16:21.470553 32174 raft_consensus.cc:482] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:16:21.470882 32174 raft_consensus.cc:504] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b1650603d4846b0ab3a75db70545cd9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37193 } } peers { permanent_uuid: "4a86881f7b5949c6b4c66b5501d511a6" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41047 } } peers { permanent_uuid: "4b8d3747e7a240acb769d8b40c9fff76" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 41967 } }
I20260811 19:16:21.471635 32174 leader_election.cc:288] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0b1650603d4846b0ab3a75db70545cd9 (127.3.61.131:37193), 4a86881f7b5949c6b4c66b5501d511a6 (127.3.61.130:41047)
W20260811 19:16:21.472788 31508 leader_election.cc:341] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0b1650603d4846b0ab3a75db70545cd9 (127.3.61.131:37193): Not found: Tablet not found: f2540d657eea4171b930aa7bc8e79bcb
W20260811 19:16:21.473017 31508 leader_election.cc:341] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a86881f7b5949c6b4c66b5501d511a6 (127.3.61.130:41047): Not found: Tablet not found: f2540d657eea4171b930aa7bc8e79bcb
I20260811 19:16:21.473133 31508 leader_election.cc:302] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b8d3747e7a240acb769d8b40c9fff76; no voters: 0b1650603d4846b0ab3a75db70545cd9, 4a86881f7b5949c6b4c66b5501d511a6
I20260811 19:16:21.473479 32174 raft_consensus.cc:2726] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 19:16:21.862694 32175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:21.861719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43283 (local address 127.3.61.130:35789)
0811 19:16:21.862378 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:21.862505 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:43283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":390,"threads_started":1}
I20260811 19:16:22.326215 32176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:22.325298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43283 (local address 127.3.61.129:60093)
0811 19:16:22.325912 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:22.326015 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:43283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":266,"threads_started":1}
I20260811 19:16:22.392338 32177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:22.391080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43283 (local address 127.3.61.131:59747)
0811 19:16:22.392006 (+ 926us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:22.392112 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:43283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":815,"thread_start_us":306,"threads_started":1}
I20260811 19:16:22.864926 32180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:22.863979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43283 (local address 127.3.61.130:45959)
0811 19:16:22.864632 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:22.864721 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:43283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":421,"threads_started":1}
I20260811 19:16:23.328289 32181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:23.327378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43283 (local address 127.3.61.129:46623)
0811 19:16:23.327956 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:23.328067 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:43283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":347,"threads_started":1}
I20260811 19:16:23.394213 32182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:23.393361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43283 (local address 127.3.61.131:56459)
0811 19:16:23.393938 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:23.394022 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:43283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":353,"threads_started":1}
I20260811 19:16:23.866991 32183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:23.866081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43283 (local address 127.3.61.130:59593)
0811 19:16:23.866679 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:23.866769 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:43283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":288,"threads_started":1}
I20260811 19:16:24.330132 32184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:24.329304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43283 (local address 127.3.61.129:46995)
0811 19:16:24.329839 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:24.329942 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:43283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":257,"threads_started":1}
I20260811 19:16:24.396859 32191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:24.395630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43283 (local address 127.3.61.131:44881)
0811 19:16:24.396263 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:24.396358 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:43283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":295,"threads_started":1}
I20260811 19:16:24.868719 32193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:24.867957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43283 (local address 127.3.61.130:52523)
0811 19:16:24.868435 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:24.868527 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:43283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":235,"threads_started":1}
I20260811 19:16:25.332011 32195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:25.331212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43283 (local address 127.3.61.129:43459)
0811 19:16:25.331724 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:25.331826 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:43283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":276,"threads_started":1}
I20260811 19:16:25.399542 32196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:25.398203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43283 (local address 127.3.61.131:48551)
0811 19:16:25.398756 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:25.398855 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:43283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":249,"threads_started":1}
I20260811 19:16:25.755611 32199 raft_consensus.cc:482] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:16:25.755918 32199 raft_consensus.cc:504] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b1650603d4846b0ab3a75db70545cd9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37193 } } peers { permanent_uuid: "4a86881f7b5949c6b4c66b5501d511a6" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41047 } } peers { permanent_uuid: "4b8d3747e7a240acb769d8b40c9fff76" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 41967 } }
I20260811 19:16:25.756633 32199 leader_election.cc:288] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0b1650603d4846b0ab3a75db70545cd9 (127.3.61.131:37193), 4a86881f7b5949c6b4c66b5501d511a6 (127.3.61.130:41047)
W20260811 19:16:25.757767 31508 leader_election.cc:341] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0b1650603d4846b0ab3a75db70545cd9 (127.3.61.131:37193): Not found: Tablet not found: f2540d657eea4171b930aa7bc8e79bcb
W20260811 19:16:25.758370 31508 leader_election.cc:341] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a86881f7b5949c6b4c66b5501d511a6 (127.3.61.130:41047): Not found: Tablet not found: f2540d657eea4171b930aa7bc8e79bcb
I20260811 19:16:25.758562 31508 leader_election.cc:302] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b8d3747e7a240acb769d8b40c9fff76; no voters: 0b1650603d4846b0ab3a75db70545cd9, 4a86881f7b5949c6b4c66b5501d511a6
I20260811 19:16:25.758960 32199 raft_consensus.cc:2726] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 19:16:25.870651 32200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:25.869835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43283 (local address 127.3.61.130:49897)
0811 19:16:25.870371 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:25.870474 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:43283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":250,"threads_started":1}
I20260811 19:16:26.334148 32202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:26.333286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43283 (local address 127.3.61.129:53915)
0811 19:16:26.333854 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:26.333948 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:43283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":278,"threads_started":1}
I20260811 19:16:26.402038 32203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:26.400898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43283 (local address 127.3.61.131:36451)
0811 19:16:26.401636 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:26.401811 (+ 175us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:43283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":357,"threads_started":1}
I20260811 19:16:26.872661 32205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:26.871812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43283 (local address 127.3.61.130:46131)
0811 19:16:26.872369 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:26.872464 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:43283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":307,"threads_started":1}
I20260811 19:16:27.336136 32207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:27.335294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43283 (local address 127.3.61.129:48475)
0811 19:16:27.335854 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:27.335952 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:43283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":308,"threads_started":1}
I20260811 19:16:27.404156 32208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:27.403336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43283 (local address 127.3.61.131:41191)
0811 19:16:27.403865 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:27.403961 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:43283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":278,"threads_started":1}
I20260811 19:16:27.874847 32209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:27.874033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43283 (local address 127.3.61.130:49717)
0811 19:16:27.874566 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:27.874670 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:43283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":276,"threads_started":1}
I20260811 19:16:28.338259 32214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:28.337433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43283 (local address 127.3.61.129:54161)
0811 19:16:28.337996 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:28.338088 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:43283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":311,"threads_started":1}
I20260811 19:16:28.406162 32215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:28.405185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43283 (local address 127.3.61.131:45211)
0811 19:16:28.405836 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:28.405961 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:43283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":305,"threads_started":1}
I20260811 19:16:28.877308 32216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:28.876103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43283 (local address 127.3.61.130:54457)
0811 19:16:28.876900 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:28.877051 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:43283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":306,"threads_started":1}
I20260811 19:16:29.340687 32217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:29.339712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43283 (local address 127.3.61.129:52937)
0811 19:16:29.340395 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:29.340497 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:43283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":370,"threads_started":1}
I20260811 19:16:29.408694 32218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:29.407391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43283 (local address 127.3.61.131:56615)
0811 19:16:29.408099 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:29.408246 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:43283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":343,"threads_started":1}
I20260811 19:16:29.879782 32220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:29.878686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43283 (local address 127.3.61.130:39233)
0811 19:16:29.879462 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:29.879584 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:43283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":656,"thread_start_us":439,"threads_started":1}
I20260811 19:16:30.343995 32222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:30.341987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43283 (local address 127.3.61.129:54253)
0811 19:16:30.342675 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:30.342808 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:43283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":289,"threads_started":1}
I20260811 19:16:30.411121 32223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:30.410014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43283 (local address 127.3.61.131:35339)
0811 19:16:30.410735 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:30.410875 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:43283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":363,"threads_started":1}
I20260811 19:16:30.620026 32224 raft_consensus.cc:482] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:16:30.620253 32224 raft_consensus.cc:504] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b1650603d4846b0ab3a75db70545cd9" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 37193 } } peers { permanent_uuid: "4a86881f7b5949c6b4c66b5501d511a6" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41047 } } peers { permanent_uuid: "4b8d3747e7a240acb769d8b40c9fff76" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 41967 } }
I20260811 19:16:30.621775 32224 leader_election.cc:288] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0b1650603d4846b0ab3a75db70545cd9 (127.3.61.131:37193), 4a86881f7b5949c6b4c66b5501d511a6 (127.3.61.130:41047)
W20260811 19:16:30.623317 31508 leader_election.cc:341] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a86881f7b5949c6b4c66b5501d511a6 (127.3.61.130:41047): Not found: Tablet not found: f2540d657eea4171b930aa7bc8e79bcb
W20260811 19:16:30.623723 31508 leader_election.cc:341] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0b1650603d4846b0ab3a75db70545cd9 (127.3.61.131:37193): Not found: Tablet not found: f2540d657eea4171b930aa7bc8e79bcb
I20260811 19:16:30.623847 31508 leader_election.cc:302] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b8d3747e7a240acb769d8b40c9fff76; no voters: 0b1650603d4846b0ab3a75db70545cd9, 4a86881f7b5949c6b4c66b5501d511a6
I20260811 19:16:30.624157 32224 raft_consensus.cc:2726] T f2540d657eea4171b930aa7bc8e79bcb P 4b8d3747e7a240acb769d8b40c9fff76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 19:16:30.882328 32225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:30.881263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43283 (local address 127.3.61.130:57893)
0811 19:16:30.881980 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:30.882100 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:43283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":369,"threads_started":1}
I20260811 19:16:31.346614 32229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:31.345823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43283 (local address 127.3.61.129:46239)
0811 19:16:31.346321 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:31.346424 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:43283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":264,"threads_started":1}
I20260811 19:16:31.413056 32230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:31.412134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43283 (local address 127.3.61.131:52101)
0811 19:16:31.412719 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:31.412830 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:43283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":312,"threads_started":1}
I20260811 19:16:31.884984 32232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:31.883985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43283 (local address 127.3.61.130:49301)
0811 19:16:31.884614 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:31.884770 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:43283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":299,"threads_started":1}
I20260811 19:16:32.349058 32233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:32.348045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43283 (local address 127.3.61.129:50181)
0811 19:16:32.348749 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:32.348865 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:43283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":355,"threads_started":1}
I20260811 19:16:32.388101 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 31413
I20260811 19:16:32.406762 32235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:32.389671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50018 (local address 127.3.61.129:41967)
0811 19:16:32.390329 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:32.390340 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:32.390374 (+ 34us) server_negotiation.cc:407] Connection header received
0811 19:16:32.390473 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:32.390485 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:32.390782 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:32.390944 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:16:32.392483 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:32.393541 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:32.394883 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:32.395615 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:32.395727 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:32.398835 (+ 3108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:16:32.398851 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:16:32.398870 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:16:32.398916 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:16:32.402657 (+ 3741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:16:32.403591 (+ 934us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:16:32.403600 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:16:32.403603 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:16:32.403801 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:16:32.404222 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:16:32.404226 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:16:32.404228 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:16:32.404862 (+ 634us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:16:32.405375 (+ 513us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:32.405953 (+ 578us) server_negotiation.cc:299] Negotiation successful
0811 19:16:32.406392 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":266,"threads_started":1}
I20260811 19:16:32.408200 31696 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:16:32.415213 32237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:32.414360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:43283 (local address 127.3.61.131:50975)
0811 19:16:32.414913 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:32.415046 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.190:43283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":284,"threads_started":1}
I20260811 19:16:32.506454 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 31413
I20260811 19:16:32.532229 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 31712
I20260811 19:16:32.548360 32238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:32.533072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42834 (local address 127.3.61.130:41047)
0811 19:16:32.533529 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:32.533533 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:32.533551 (+ 18us) server_negotiation.cc:407] Connection header received
0811 19:16:32.533610 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:32.533614 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:32.533706 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:32.533797 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:16:32.535350 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:32.536173 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:32.537955 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:32.538583 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:32.538754 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:32.542113 (+ 3359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:16:32.542121 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:16:32.542125 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:16:32.542173 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:16:32.545294 (+ 3121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:16:32.546227 (+ 933us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:16:32.546232 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:16:32.546235 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:16:32.546400 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:16:32.546840 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:16:32.546845 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:16:32.546847 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:16:32.547205 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:16:32.547460 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:32.547841 (+ 381us) server_negotiation.cc:299] Negotiation successful
0811 19:16:32.548052 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":241,"threads_started":1}
I20260811 19:16:32.549144 31909 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:16:32.639654 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 31712
I20260811 19:16:32.667440 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 31920
I20260811 19:16:32.682988 32240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:32.668300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58000 (local address 127.3.61.131:37193)
0811 19:16:32.668911 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:32.668915 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:32.668931 (+ 16us) server_negotiation.cc:407] Connection header received
0811 19:16:32.668986 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:32.668990 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:32.669060 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:32.669159 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:16:32.670801 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:32.671727 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:32.673063 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:32.673515 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:32.673632 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:32.676445 (+ 2813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:16:32.676451 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:16:32.676454 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:16:32.676494 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:16:32.679530 (+ 3036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:16:32.680763 (+ 1233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:16:32.680768 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:16:32.680771 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:16:32.680941 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:16:32.681394 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:16:32.681398 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:16:32.681401 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:16:32.681753 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:16:32.682048 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:32.682362 (+ 314us) server_negotiation.cc:299] Negotiation successful
0811 19:16:32.682599 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":338,"threads_started":1}
I20260811 19:16:32.683761 32135 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:16:32.774905 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 31920
2026-08-11T19:16:32Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (66386 ms)
[----------] 30 tests from SecurityITest (337877 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:16:32 dist-test-slave-t7g5 krb5kdc[32245](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:16:32 dist-test-slave-t7g5 krb5kdc[32245](info): set up 2 sockets
Aug 11 19:16:32 dist-test-slave-t7g5 krb5kdc[32245](info): commencing operation
krb5kdc: starting...
W20260811 19:16:34.095980 3318 mini_kdc.cc:121] Time spent starting KDC: real 1.268s user 0.000s sys 0.061s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Aug 11 19:16:34 dist-test-slave-t7g5 krb5kdc[32245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475794, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:16:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:16:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.61.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab.
Entry for principal kudu/127.3.61.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab.
I20260811 19:16:34.482355 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:35205
--webserver_interface=127.3.61.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:38543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.61.190:35205
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:16:34.875490 32270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:16:34.875862 32270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:16:34.875996 32270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:16:34.883764 32270 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:16:34.883903 32270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:16:34.883952 32270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:16:34.884043 32270 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:16:34.884306 32270 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:16:34.897325 32270 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:38543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.61.190:35205
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:35205
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.61.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:16:34.899439 32270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:16:34.900882 32270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:16:34.914115 32276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:16:34.914186 32275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:16:34.914160 32278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:16:34.914914 32270 server_base.cc:1034] running on GCE node
I20260811 19:16:34.915861 32270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:16:34.917510 32270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:16:34.918764 32270 hybrid_clock.cc:648] HybridClock initialized: now 1786475794918720 us; error 55 us; skew 500 ppm
I20260811 19:16:34.919157 32270 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:16:34 dist-test-slave-t7g5 krb5kdc[32245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475794, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:16:34.927973 32270 init.cc:377] Logged in from keytab as kudu/127.3.61.190@KRBTEST.COM (short username kudu)
I20260811 19:16:34.929795 32270 webserver.cc:460] Webserver started at http://127.3.61.190:37629/ using document root <none> and password file <none>
I20260811 19:16:34.930591 32270 fs_manager.cc:359] Metadata directory not provided
I20260811 19:16:34.930732 32270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:16:34.931069 32270 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:16:34.934520 32270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/master-0/data/instance:
uuid: "13997ebd774f4dde9bde004c768ea3eb"
format_stamp: "Formatted at 2026-08-11 19:16:34 on dist-test-slave-t7g5"
server_key: "bbcab6f35f16dc2445df12b2d8b72557"
server_key_iv: "b3c3aa17630f8d13331ab6649419e110"
server_key_version: "encryptionkey@0"
I20260811 19:16:34.935336 32270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/master-0/wal/instance:
uuid: "13997ebd774f4dde9bde004c768ea3eb"
format_stamp: "Formatted at 2026-08-11 19:16:34 on dist-test-slave-t7g5"
server_key: "bbcab6f35f16dc2445df12b2d8b72557"
server_key_iv: "b3c3aa17630f8d13331ab6649419e110"
server_key_version: "encryptionkey@0"
I20260811 19:16:34.941376 32270 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260811 19:16:34.945524 32289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:34.947216 32270 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 19:16:34.947422 32270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/master-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/master-0/wal
uuid: "13997ebd774f4dde9bde004c768ea3eb"
format_stamp: "Formatted at 2026-08-11 19:16:34 on dist-test-slave-t7g5"
server_key: "bbcab6f35f16dc2445df12b2d8b72557"
server_key_iv: "b3c3aa17630f8d13331ab6649419e110"
server_key_version: "encryptionkey@0"
I20260811 19:16:34.947633 32270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:16:34.979964 32270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:16:34.984519 32270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:16:34.984818 32270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:16:35.004859 32270 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.190:35205
I20260811 19:16:35.004922 32384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.190:35205 every 8 connection(s)
I20260811 19:16:35.006415 32270 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
I20260811 19:16:35.010848 32385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:16:35.012249 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 32270
I20260811 19:16:35.012528 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/master-0/wal/instance
I20260811 19:16:35.013443 3318 external_mini_cluster.cc:1428] Setting key 91e09cd9753cf60e6ff53898f29d0f7d
I20260811 19:16:35.025597 32385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13997ebd774f4dde9bde004c768ea3eb: Bootstrap starting.
I20260811 19:16:35.030445 32385 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 13997ebd774f4dde9bde004c768ea3eb: Neither blocks nor log segments found. Creating new log.
I20260811 19:16:35.031864 32385 log.cc:824] T 00000000000000000000000000000000 P 13997ebd774f4dde9bde004c768ea3eb: Log is configured to *not* fsync() on all Append() calls
Aug 11 19:16:35 dist-test-slave-t7g5 krb5kdc[32245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475794, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:16:35.035135 32385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13997ebd774f4dde9bde004c768ea3eb: No bootstrap required, opened a new log
I20260811 19:16:35.043349 32391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:35.016487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38092 (local address 127.3.61.190:35205)
0811 19:16:35.017803 (+ 1316us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:35.017825 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:35.017889 (+ 64us) server_negotiation.cc:407] Connection header received
0811 19:16:35.019802 (+ 1913us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:35.019829 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:35.020272 (+ 443us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:35.020630 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:16:35.022741 (+ 2111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:35.024150 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:35.026817 (+ 2667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:35.027760 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:35.027952 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:35.033336 (+ 5384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:16:35.033346 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:16:35.033365 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:16:35.033417 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:16:35.037705 (+ 4288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:16:35.038786 (+ 1081us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:16:35.038796 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:16:35.038806 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:16:35.039037 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:16:35.039500 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:16:35.039505 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:16:35.039507 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:16:35.040311 (+ 804us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:16:35.040692 (+ 381us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:35.041252 (+ 560us) server_negotiation.cc:299] Negotiation successful
0811 19:16:35.041893 (+ 641us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":874,"thread_start_us":516,"threads_started":1}
I20260811 19:16:35.044608 32385 raft_consensus.cc:348] T 00000000000000000000000000000000 P 13997ebd774f4dde9bde004c768ea3eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13997ebd774f4dde9bde004c768ea3eb" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35205 } }
I20260811 19:16:35.045130 32385 raft_consensus.cc:374] T 00000000000000000000000000000000 P 13997ebd774f4dde9bde004c768ea3eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:16:35.045271 32385 raft_consensus.cc:729] T 00000000000000000000000000000000 P 13997ebd774f4dde9bde004c768ea3eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13997ebd774f4dde9bde004c768ea3eb, State: Initialized, Role: FOLLOWER
I20260811 19:16:35.046325 32385 consensus_queue.cc:260] T 00000000000000000000000000000000 P 13997ebd774f4dde9bde004c768ea3eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13997ebd774f4dde9bde004c768ea3eb" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35205 } }
I20260811 19:16:35.047216 32385 raft_consensus.cc:388] T 00000000000000000000000000000000 P 13997ebd774f4dde9bde004c768ea3eb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:16:35.047420 32385 raft_consensus.cc:482] T 00000000000000000000000000000000 P 13997ebd774f4dde9bde004c768ea3eb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:16:35.047652 32385 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 13997ebd774f4dde9bde004c768ea3eb [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:16:35.050316 32385 raft_consensus.cc:504] T 00000000000000000000000000000000 P 13997ebd774f4dde9bde004c768ea3eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13997ebd774f4dde9bde004c768ea3eb" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35205 } }
I20260811 19:16:35.050923 32385 leader_election.cc:302] T 00000000000000000000000000000000 P 13997ebd774f4dde9bde004c768ea3eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 13997ebd774f4dde9bde004c768ea3eb; no voters:
I20260811 19:16:35.051613 32385 leader_election.cc:288] T 00000000000000000000000000000000 P 13997ebd774f4dde9bde004c768ea3eb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:16:35.051750 32393 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 13997ebd774f4dde9bde004c768ea3eb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:16:35.052543 32393 raft_consensus.cc:686] T 00000000000000000000000000000000 P 13997ebd774f4dde9bde004c768ea3eb [term 1 LEADER]: Becoming Leader. State: Replica: 13997ebd774f4dde9bde004c768ea3eb, State: Running, Role: LEADER
I20260811 19:16:35.053220 32393 consensus_queue.cc:237] T 00000000000000000000000000000000 P 13997ebd774f4dde9bde004c768ea3eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13997ebd774f4dde9bde004c768ea3eb" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35205 } }
I20260811 19:16:35.054068 32385 sys_catalog.cc:564] T 00000000000000000000000000000000 P 13997ebd774f4dde9bde004c768ea3eb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:16:35.056416 32397 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13997ebd774f4dde9bde004c768ea3eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "13997ebd774f4dde9bde004c768ea3eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13997ebd774f4dde9bde004c768ea3eb" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35205 } } }
I20260811 19:16:35.056967 32397 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13997ebd774f4dde9bde004c768ea3eb [sys.catalog]: This master's current role is: LEADER
I20260811 19:16:35.057279 32398 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13997ebd774f4dde9bde004c768ea3eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 13997ebd774f4dde9bde004c768ea3eb. Latest consensus state: current_term: 1 leader_uuid: "13997ebd774f4dde9bde004c768ea3eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13997ebd774f4dde9bde004c768ea3eb" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35205 } } }
I20260811 19:16:35.057611 32398 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13997ebd774f4dde9bde004c768ea3eb [sys.catalog]: This master's current role is: LEADER
I20260811 19:16:35.062678 32406 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:16:35.068856 32406 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:16:35.081965 32406 catalog_manager.cc:1347] Generated new cluster ID: 3c0f1092749f4d4292e74a466836a4eb
I20260811 19:16:35.082264 32406 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:16:35.101874 32406 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:16:35.103950 32406 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:16:35.112493 32406 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 13997ebd774f4dde9bde004c768ea3eb: Generated new TSK 0
I20260811 19:16:35.113731 32406 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.61.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab.
Entry for principal kudu/127.3.61.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab.
I20260811 19:16:35.213559 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.129:0
--local_ip_for_outbound_sockets=127.3.61.129
--webserver_interface=127.3.61.129
--webserver_port=0
--tserver_master_addrs=127.3.61.190:35205
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab
--principal=kudu/127.3.61.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:38543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:16:35.612823 32428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:16:35.613152 32428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:16:35.613287 32428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:16:35.621738 32428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:16:35.621923 32428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:16:35.622090 32428 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.129
I20260811 19:16:35.633708 32428 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:38543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab
--principal=kudu/127.3.61.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.61.129
--webserver_port=0
--tserver_master_addrs=127.3.61.190:35205
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.129
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:16:35.635170 32428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:16:35.636566 32428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:16:35.653596 32438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:16:35.653710 32439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:16:35.653680 32442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:16:35.654639 32428 server_base.cc:1034] running on GCE node
I20260811 19:16:35.655347 32428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:16:35.656438 32428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:16:35.657732 32428 hybrid_clock.cc:648] HybridClock initialized: now 1786475795657613 us; error 66 us; skew 500 ppm
I20260811 19:16:35.658128 32428 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:16:35 dist-test-slave-t7g5 krb5kdc[32245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475795, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:16:35.662482 32428 init.cc:377] Logged in from keytab as kudu/127.3.61.129@KRBTEST.COM (short username kudu)
I20260811 19:16:35.664636 32428 webserver.cc:460] Webserver started at http://127.3.61.129:39243/ using document root <none> and password file <none>
I20260811 19:16:35.665426 32428 fs_manager.cc:359] Metadata directory not provided
I20260811 19:16:35.665565 32428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:16:35.665936 32428 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:16:35.669461 32428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/ts-0/data/instance:
uuid: "75742c8598794d92a22bdc0dfe029252"
format_stamp: "Formatted at 2026-08-11 19:16:35 on dist-test-slave-t7g5"
server_key: "4920103de2500ff9596748090fba050e"
server_key_iv: "762318b6720156536ddb9a92c4d197e0"
server_key_version: "encryptionkey@0"
I20260811 19:16:35.670385 32428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/ts-0/wal/instance:
uuid: "75742c8598794d92a22bdc0dfe029252"
format_stamp: "Formatted at 2026-08-11 19:16:35 on dist-test-slave-t7g5"
server_key: "4920103de2500ff9596748090fba050e"
server_key_iv: "762318b6720156536ddb9a92c4d197e0"
server_key_version: "encryptionkey@0"
I20260811 19:16:35.676620 32428 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 19:16:35.682572 32458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:35.684175 32428 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 19:16:35.684351 32428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/ts-0/wal
uuid: "75742c8598794d92a22bdc0dfe029252"
format_stamp: "Formatted at 2026-08-11 19:16:35 on dist-test-slave-t7g5"
server_key: "4920103de2500ff9596748090fba050e"
server_key_iv: "762318b6720156536ddb9a92c4d197e0"
server_key_version: "encryptionkey@0"
I20260811 19:16:35.684563 32428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:16:35.713074 32428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:16:35.718065 32428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:16:35.718374 32428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:16:35.719452 32428 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:16:35.721683 32428 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:16:35.721812 32428 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260811 19:16:35.721956 32428 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:16:35.722016 32428 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:35.759948 32428 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.129:46815
I20260811 19:16:35.760133 32655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.129:46815 every 8 connection(s)
I20260811 19:16:35.761359 32428 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
I20260811 19:16:35.767233 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 32428
I20260811 19:16:35.767508 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/ts-0/wal/instance
I20260811 19:16:35.768391 3318 external_mini_cluster.cc:1428] Setting key 630a3a17c87a25d3734d622325902f24
Aug 11 19:16:35 dist-test-slave-t7g5 krb5kdc[32245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475795, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.129@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:16:35.788014 32663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:35.764860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:38215 (local address 127.3.61.190:35205)
0811 19:16:35.765379 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:35.765385 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:35.766538 (+ 1153us) server_negotiation.cc:407] Connection header received
0811 19:16:35.769302 (+ 2764us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:35.769309 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:35.769482 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:35.769636 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:16:35.771867 (+ 2231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:35.772636 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:35.775266 (+ 2630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:35.775732 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:35.775849 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:35.780764 (+ 4915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:16:35.780770 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:16:35.780774 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:16:35.780816 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:16:35.783915 (+ 3099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:16:35.784936 (+ 1021us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:16:35.784941 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:16:35.784944 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:16:35.785091 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:16:35.785583 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:16:35.785588 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:16:35.785590 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:16:35.785970 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:16:35.786125 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:35.787063 (+ 938us) server_negotiation.cc:299] Negotiation successful
0811 19:16:35.787456 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":270,"threads_started":1}
I20260811 19:16:35.789393 32664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:35.765461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35205 (local address 127.3.61.129:38215)
0811 19:16:35.766361 (+ 900us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:35.766427 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 19:16:35.768935 (+ 2508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:16:35.769985 (+ 1050us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:16:35.769995 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:16:35.770684 (+ 689us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:16:35.771629 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:35.771664 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:35.772842 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:35.772849 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:35.775057 (+ 2208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:35.775084 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:35.775983 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:35.775991 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:35.776284 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:35.777095 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:16:35.777125 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:16:35.780307 (+ 3182us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:16:35.784150 (+ 3843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:16:35.784161 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:16:35.784183 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:16:35.784782 (+ 599us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:16:35.785212 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:16:35.785218 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:16:35.785222 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:16:35.785453 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:16:35.786111 (+ 658us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:16:35.786129 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:16:35.786603 (+ 474us) client_negotiation.cc:769] Sending connection context
0811 19:16:35.786922 (+ 319us) client_negotiation.cc:240] Negotiation successful
0811 19:16:35.787986 (+ 1064us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":382,"threads_started":1}
I20260811 19:16:35.791343 32656 heartbeater.cc:347] Connected to a master server at 127.3.61.190:35205
I20260811 19:16:35.791687 32656 heartbeater.cc:464] Registering TS with master...
I20260811 19:16:35.792500 32656 heartbeater.cc:511] Master 127.3.61.190:35205 requested a full tablet report, sending...
I20260811 19:16:35.794915 32320 ts_manager.cc:194] Registered new tserver with Master: 75742c8598794d92a22bdc0dfe029252 (127.3.61.129:46815)
I20260811 19:16:35.796451 32320 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.129@KRBTEST.COM'} at 127.3.61.129:38215
WARNING: no policy specified for kudu/127.3.61.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab.
Entry for principal kudu/127.3.61.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab.
I20260811 19:16:35.856612 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.130:0
--local_ip_for_outbound_sockets=127.3.61.130
--webserver_interface=127.3.61.130
--webserver_port=0
--tserver_master_addrs=127.3.61.190:35205
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab
--principal=kudu/127.3.61.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:38543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:16:36.263485 32670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:16:36.263839 32670 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:16:36.263949 32670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:16:36.272851 32670 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:16:36.273005 32670 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:16:36.273128 32670 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.130
I20260811 19:16:36.285461 32670 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:38543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab
--principal=kudu/127.3.61.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.61.130
--webserver_port=0
--tserver_master_addrs=127.3.61.190:35205
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.130
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:16:36.286989 32670 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:16:36.288381 32670 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:16:36.302891 32681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:16:36.303216 32689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:16:36.303498 32687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:16:36.305081 32670 server_base.cc:1034] running on GCE node
I20260811 19:16:36.305960 32670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:16:36.307361 32670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:16:36.308714 32670 hybrid_clock.cc:648] HybridClock initialized: now 1786475796308631 us; error 72 us; skew 500 ppm
I20260811 19:16:36.309185 32670 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:16:36 dist-test-slave-t7g5 krb5kdc[32245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475796, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:16:36.313057 32670 init.cc:377] Logged in from keytab as kudu/127.3.61.130@KRBTEST.COM (short username kudu)
I20260811 19:16:36.315151 32670 webserver.cc:460] Webserver started at http://127.3.61.130:39373/ using document root <none> and password file <none>
I20260811 19:16:36.316111 32670 fs_manager.cc:359] Metadata directory not provided
I20260811 19:16:36.316250 32670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:16:36.316601 32670 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:16:36.320020 32670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/ts-1/data/instance:
uuid: "37701d3e508341709eb870f719d9d65e"
format_stamp: "Formatted at 2026-08-11 19:16:36 on dist-test-slave-t7g5"
server_key: "a006f365fba1b257ab4eeaa0a1cd368c"
server_key_iv: "fbea126a1e685f428a1dd86b957144b1"
server_key_version: "encryptionkey@0"
I20260811 19:16:36.321045 32670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/ts-1/wal/instance:
uuid: "37701d3e508341709eb870f719d9d65e"
format_stamp: "Formatted at 2026-08-11 19:16:36 on dist-test-slave-t7g5"
server_key: "a006f365fba1b257ab4eeaa0a1cd368c"
server_key_iv: "fbea126a1e685f428a1dd86b957144b1"
server_key_version: "encryptionkey@0"
I20260811 19:16:36.328809 32670 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260811 19:16:36.333285 32700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:36.335076 32670 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 19:16:36.335270 32670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/ts-1/wal
uuid: "37701d3e508341709eb870f719d9d65e"
format_stamp: "Formatted at 2026-08-11 19:16:36 on dist-test-slave-t7g5"
server_key: "a006f365fba1b257ab4eeaa0a1cd368c"
server_key_iv: "fbea126a1e685f428a1dd86b957144b1"
server_key_version: "encryptionkey@0"
I20260811 19:16:36.335505 32670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:16:36.354441 32670 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:16:36.360231 32670 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:16:36.360559 32670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:16:36.361913 32670 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:16:36.364202 32670 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:16:36.364336 32670 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:36.364478 32670 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:16:36.364535 32670 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:36.402814 32670 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.130:41003
I20260811 19:16:36.402870 423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.130:41003 every 8 connection(s)
I20260811 19:16:36.404172 32670 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
I20260811 19:16:36.408797 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 32670
I20260811 19:16:36.409055 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/ts-1/wal/instance
I20260811 19:16:36.409853 3318 external_mini_cluster.cc:1428] Setting key 8a2cd94fd18b987d8164c08a8be71ca6
Aug 11 19:16:36 dist-test-slave-t7g5 krb5kdc[32245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475796, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.130@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:16:36.430240 429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:36.407385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:51019 (local address 127.3.61.190:35205)
0811 19:16:36.408057 (+ 672us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:36.408062 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:36.409435 (+ 1373us) server_negotiation.cc:407] Connection header received
0811 19:16:36.411595 (+ 2160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:36.411602 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:36.411839 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:36.412178 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:16:36.414265 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:36.415318 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:36.417066 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:36.417642 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:36.417785 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:36.422593 (+ 4808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:16:36.422601 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:16:36.422605 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:16:36.422654 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:16:36.426141 (+ 3487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:16:36.427421 (+ 1280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:16:36.427426 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:16:36.427429 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:16:36.427616 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:16:36.428167 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:16:36.428172 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:16:36.428174 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:16:36.428560 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:16:36.428710 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:36.429577 (+ 867us) server_negotiation.cc:299] Negotiation successful
0811 19:16:36.429864 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":531,"thread_start_us":337,"threads_started":1}
I20260811 19:16:36.431311 430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:36.408250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35205 (local address 127.3.61.130:51019)
0811 19:16:36.409202 (+ 952us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:36.409253 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 19:16:36.411287 (+ 2034us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:16:36.412456 (+ 1169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:16:36.412466 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:16:36.413235 (+ 769us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:16:36.414054 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:36.414076 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:36.415561 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:36.415566 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:36.416900 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:36.416915 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:36.417927 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:36.417933 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:36.418167 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:36.419116 (+ 949us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:16:36.419155 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:16:36.422284 (+ 3129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:16:36.426397 (+ 4113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:16:36.426411 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:16:36.426436 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:16:36.427181 (+ 745us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:16:36.427732 (+ 551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:16:36.427736 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:16:36.427739 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:16:36.428029 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:16:36.428736 (+ 707us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:16:36.428747 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:16:36.429169 (+ 422us) client_negotiation.cc:769] Sending connection context
0811 19:16:36.429447 (+ 278us) client_negotiation.cc:240] Negotiation successful
0811 19:16:36.430114 (+ 667us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"thread_start_us":392,"threads_started":1}
I20260811 19:16:36.433213 425 heartbeater.cc:347] Connected to a master server at 127.3.61.190:35205
I20260811 19:16:36.433548 425 heartbeater.cc:464] Registering TS with master...
I20260811 19:16:36.434370 425 heartbeater.cc:511] Master 127.3.61.190:35205 requested a full tablet report, sending...
I20260811 19:16:36.436072 32320 ts_manager.cc:194] Registered new tserver with Master: 37701d3e508341709eb870f719d9d65e (127.3.61.130:41003)
I20260811 19:16:36.437059 32320 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.130@KRBTEST.COM'} at 127.3.61.130:51019
WARNING: no policy specified for kudu/127.3.61.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab.
Entry for principal kudu/127.3.61.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab.
I20260811 19:16:36.491708 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.131:0
--local_ip_for_outbound_sockets=127.3.61.131
--webserver_interface=127.3.61.131
--webserver_port=0
--tserver_master_addrs=127.3.61.190:35205
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab
--principal=kudu/127.3.61.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:38543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:16:36.799865 32656 heartbeater.cc:503] Master 127.3.61.190:35205 was elected leader, sending a full tablet report...
W20260811 19:16:36.882896 434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:16:36.883370 434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:16:36.883558 434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:16:36.891444 434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:16:36.891594 434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:16:36.891764 434 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.131
I20260811 19:16:36.904597 434 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:38543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab
--principal=kudu/127.3.61.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.61.131
--webserver_port=0
--tserver_master_addrs=127.3.61.190:35205
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.131
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:16:36.906185 434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:16:36.907734 434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:16:36.921545 445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:16:36.921500 448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:16:36.921578 444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:16:36.922745 434 server_base.cc:1034] running on GCE node
I20260811 19:16:36.923492 434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:16:36.924568 434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:16:36.925791 434 hybrid_clock.cc:648] HybridClock initialized: now 1786475796925720 us; error 48 us; skew 500 ppm
I20260811 19:16:36.926160 434 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:16:36 dist-test-slave-t7g5 krb5kdc[32245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475796, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:16:36.929626 434 init.cc:377] Logged in from keytab as kudu/127.3.61.131@KRBTEST.COM (short username kudu)
I20260811 19:16:36.931437 434 webserver.cc:460] Webserver started at http://127.3.61.131:43139/ using document root <none> and password file <none>
I20260811 19:16:36.932215 434 fs_manager.cc:359] Metadata directory not provided
I20260811 19:16:36.932334 434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:16:36.932623 434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:16:36.935876 434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/ts-2/data/instance:
uuid: "a84233a301c3424fab33c25d4cfd771f"
format_stamp: "Formatted at 2026-08-11 19:16:36 on dist-test-slave-t7g5"
server_key: "4f96f7229004245fe4e8a204cfbd49d9"
server_key_iv: "ade6873b7d37203322959a3019575880"
server_key_version: "encryptionkey@0"
I20260811 19:16:36.936868 434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/ts-2/wal/instance:
uuid: "a84233a301c3424fab33c25d4cfd771f"
format_stamp: "Formatted at 2026-08-11 19:16:36 on dist-test-slave-t7g5"
server_key: "4f96f7229004245fe4e8a204cfbd49d9"
server_key_iv: "ade6873b7d37203322959a3019575880"
server_key_version: "encryptionkey@0"
I20260811 19:16:36.943831 434 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260811 19:16:36.948619 456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:36.950544 434 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 19:16:36.950735 434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/ts-2/wal
uuid: "a84233a301c3424fab33c25d4cfd771f"
format_stamp: "Formatted at 2026-08-11 19:16:36 on dist-test-slave-t7g5"
server_key: "4f96f7229004245fe4e8a204cfbd49d9"
server_key_iv: "ade6873b7d37203322959a3019575880"
server_key_version: "encryptionkey@0"
I20260811 19:16:36.950971 434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:16:36.988574 434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:16:36.993893 434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:16:36.994195 434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:16:36.995240 434 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:16:36.996735 434 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:16:36.996820 434 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:36.996927 434 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:16:36.996974 434 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:37.034531 434 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.131:45715
I20260811 19:16:37.034582 639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.131:45715 every 8 connection(s)
I20260811 19:16:37.035919 434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
I20260811 19:16:37.041738 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 434
I20260811 19:16:37.041951 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/ts-2/wal/instance
I20260811 19:16:37.042583 3318 external_mini_cluster.cc:1428] Setting key 65bcdd08ba2e0e75cec2882ee59763f3
Aug 11 19:16:37 dist-test-slave-t7g5 krb5kdc[32245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475796, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.131@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:16:37.062083 643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:37.039135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:46289 (local address 127.3.61.190:35205)
0811 19:16:37.040381 (+ 1246us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:37.040388 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:37.041952 (+ 1564us) server_negotiation.cc:407] Connection header received
0811 19:16:37.044350 (+ 2398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:37.044367 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:37.044498 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:37.044659 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:16:37.046783 (+ 2124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:37.047771 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:37.049765 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:37.050285 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:37.050407 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:37.054870 (+ 4463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:16:37.054875 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:16:37.054879 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:16:37.054929 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:16:37.057935 (+ 3006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:16:37.058851 (+ 916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:16:37.058855 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:16:37.058857 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:16:37.058989 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:16:37.059838 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:16:37.059843 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:16:37.059846 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:16:37.060335 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:16:37.060549 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:37.061484 (+ 935us) server_negotiation.cc:299] Negotiation successful
0811 19:16:37.061757 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1104,"thread_start_us":912,"threads_started":1}
I20260811 19:16:37.063004 644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:37.040749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35205 (local address 127.3.61.131:46289)
0811 19:16:37.041645 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:37.041729 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0811 19:16:37.043906 (+ 2177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:16:37.044987 (+ 1081us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:16:37.045000 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:16:37.045915 (+ 915us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:16:37.046567 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:37.046586 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:37.048085 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:37.048089 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:37.049529 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:37.049545 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:37.050552 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:37.050559 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:37.050786 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:37.051548 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:16:37.051579 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:16:37.054552 (+ 2973us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:16:37.058095 (+ 3543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:16:37.058106 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:16:37.058123 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:16:37.058709 (+ 586us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:16:37.059166 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:16:37.059171 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:16:37.059174 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:16:37.059672 (+ 498us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:16:37.060558 (+ 886us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:16:37.060568 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:16:37.061045 (+ 477us) client_negotiation.cc:769] Sending connection context
0811 19:16:37.061321 (+ 276us) client_negotiation.cc:240] Negotiation successful
0811 19:16:37.061832 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":361,"threads_started":1}
I20260811 19:16:37.064553 640 heartbeater.cc:347] Connected to a master server at 127.3.61.190:35205
I20260811 19:16:37.064898 640 heartbeater.cc:464] Registering TS with master...
I20260811 19:16:37.066215 640 heartbeater.cc:511] Master 127.3.61.190:35205 requested a full tablet report, sending...
I20260811 19:16:37.068437 32320 ts_manager.cc:194] Registered new tserver with Master: a84233a301c3424fab33c25d4cfd771f (127.3.61.131:45715)
I20260811 19:16:37.069444 32320 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.131@KRBTEST.COM'} at 127.3.61.131:46289
I20260811 19:16:37.074055 3318 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 19:16:37.094786 643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:37.079882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38106 (local address 127.3.61.190:35205)
0811 19:16:37.080070 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:37.080075 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:37.080746 (+ 671us) server_negotiation.cc:407] Connection header received
0811 19:16:37.080938 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:37.080943 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:37.081022 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:37.081111 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:16:37.082802 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:37.083568 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:37.085311 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:37.085870 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:37.086004 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:37.088995 (+ 2991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:16:37.089002 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:16:37.089005 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:16:37.089047 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:16:37.091935 (+ 2888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:16:37.092657 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:16:37.092663 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:16:37.092666 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:16:37.092794 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:16:37.093173 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:16:37.093178 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:16:37.093181 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:16:37.093557 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:16:37.093821 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:37.094216 (+ 395us) server_negotiation.cc:299] Negotiation successful
0811 19:16:37.094471 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 19:16:37.099742 32320 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38106:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 19:16:37.107800 32320 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:16:37.137223 663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:37.125949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51780 (local address 127.3.61.129:46815)
0811 19:16:37.127687 (+ 1738us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:37.127696 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:37.128488 (+ 792us) server_negotiation.cc:407] Connection header received
0811 19:16:37.128682 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:37.128690 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:37.128967 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:37.129175 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:16:37.130104 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:37.131784 (+ 1680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:37.134684 (+ 2900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:37.135998 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:37.136127 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:37.136388 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:37.136708 (+ 320us) server_negotiation.cc:299] Negotiation successful
0811 19:16:37.136896 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1524,"thread_start_us":1314,"threads_started":1}
I20260811 19:16:37.137279 662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:37.124934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:46815 (local address 127.0.0.1:51780)
0811 19:16:37.128343 (+ 3409us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:37.128368 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 19:16:37.128519 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:16:37.129426 (+ 907us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:16:37.129431 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:16:37.129451 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:16:37.129883 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:37.129898 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:37.131979 (+ 2081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:37.131985 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:37.134475 (+ 2490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:37.134494 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:37.136216 (+ 1722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:37.136223 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:37.136458 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:37.136504 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 19:16:37.136617 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 19:16:37.136822 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3267,"mutex_wait_us":89,"thread_start_us":3037,"threads_started":1}
I20260811 19:16:37.137686 661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:37.124721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58552 (local address 127.3.61.131:45715)
0811 19:16:37.126606 (+ 1885us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:37.126615 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:37.126655 (+ 40us) server_negotiation.cc:407] Connection header received
0811 19:16:37.126765 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:37.126773 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:37.127126 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:37.127344 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:16:37.128570 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:37.131784 (+ 3214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:37.134748 (+ 2964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:37.136199 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:37.136330 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:37.136570 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:37.137111 (+ 541us) server_negotiation.cc:299] Negotiation successful
0811 19:16:37.137280 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1687,"thread_start_us":2123,"threads_started":1}
I20260811 19:16:37.139111 660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:37.124234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.131:45715 (local address 127.0.0.1:58552)
0811 19:16:37.125064 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:37.125122 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 19:16:37.125349 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:16:37.127661 (+ 2312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:16:37.127670 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:16:37.127705 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:16:37.128344 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:37.128367 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:37.131933 (+ 3566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:37.131939 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:37.134541 (+ 2602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:37.134555 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:37.136718 (+ 2163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:37.136725 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:37.136892 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:37.136928 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 19:16:37.138715 (+ 1787us) client_negotiation.cc:240] Negotiation successful
0811 19:16:37.138832 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"mutex_wait_us":120,"thread_start_us":331,"threads_started":1}
I20260811 19:16:37.140553 664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:37.128814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.130:41003 (local address 127.0.0.1:56288)
0811 19:16:37.129728 (+ 914us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:37.129758 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 19:16:37.129895 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:16:37.133008 (+ 3113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:16:37.133013 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:16:37.133039 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:16:37.133532 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:37.133545 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:37.135381 (+ 1836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:37.135386 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:37.137477 (+ 2091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:37.137493 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:37.139807 (+ 2314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:37.139814 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:37.139972 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:37.140003 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 19:16:37.140092 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 19:16:37.140206 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"mutex_wait_us":5,"thread_start_us":296,"threads_started":1}
I20260811 19:16:37.141165 665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:37.129076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56288 (local address 127.3.61.130:41003)
0811 19:16:37.132336 (+ 3260us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:37.132345 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:37.132374 (+ 29us) server_negotiation.cc:407] Connection header received
0811 19:16:37.132508 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:37.132517 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:37.132773 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:37.132997 (+ 224us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:16:37.133738 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:37.135229 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:37.137707 (+ 2478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:37.138981 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:37.139626 (+ 645us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:37.139818 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:37.140807 (+ 989us) server_negotiation.cc:299] Negotiation successful
0811 19:16:37.140943 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3082,"thread_start_us":2961,"threads_started":1}
I20260811 19:16:37.141778 32541 tablet_service.cc:1467] Processing CreateTablet for tablet 3e1ac3bfe05e4d3c92209c610486ad88 (DEFAULT_TABLE table=test-table [id=1d170f49fb9a4cb582994d10c979ef5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:16:37.143409 32541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e1ac3bfe05e4d3c92209c610486ad88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:16:37.144496 325 tablet_service.cc:1467] Processing CreateTablet for tablet 3e1ac3bfe05e4d3c92209c610486ad88 (DEFAULT_TABLE table=test-table [id=1d170f49fb9a4cb582994d10c979ef5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:16:37.144845 534 tablet_service.cc:1467] Processing CreateTablet for tablet 3e1ac3bfe05e4d3c92209c610486ad88 (DEFAULT_TABLE table=test-table [id=1d170f49fb9a4cb582994d10c979ef5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:16:37.145761 325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e1ac3bfe05e4d3c92209c610486ad88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:16:37.147241 534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e1ac3bfe05e4d3c92209c610486ad88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:16:37.158289 666 tablet_bootstrap.cc:492] T 3e1ac3bfe05e4d3c92209c610486ad88 P 37701d3e508341709eb870f719d9d65e: Bootstrap starting.
I20260811 19:16:37.160612 667 tablet_bootstrap.cc:492] T 3e1ac3bfe05e4d3c92209c610486ad88 P 75742c8598794d92a22bdc0dfe029252: Bootstrap starting.
I20260811 19:16:37.162555 666 tablet_bootstrap.cc:654] T 3e1ac3bfe05e4d3c92209c610486ad88 P 37701d3e508341709eb870f719d9d65e: Neither blocks nor log segments found. Creating new log.
I20260811 19:16:37.164306 667 tablet_bootstrap.cc:654] T 3e1ac3bfe05e4d3c92209c610486ad88 P 75742c8598794d92a22bdc0dfe029252: Neither blocks nor log segments found. Creating new log.
I20260811 19:16:37.164530 666 log.cc:824] T 3e1ac3bfe05e4d3c92209c610486ad88 P 37701d3e508341709eb870f719d9d65e: Log is configured to *not* fsync() on all Append() calls
I20260811 19:16:37.165606 667 log.cc:824] T 3e1ac3bfe05e4d3c92209c610486ad88 P 75742c8598794d92a22bdc0dfe029252: Log is configured to *not* fsync() on all Append() calls
I20260811 19:16:37.166924 669 tablet_bootstrap.cc:492] T 3e1ac3bfe05e4d3c92209c610486ad88 P a84233a301c3424fab33c25d4cfd771f: Bootstrap starting.
I20260811 19:16:37.168181 666 tablet_bootstrap.cc:492] T 3e1ac3bfe05e4d3c92209c610486ad88 P 37701d3e508341709eb870f719d9d65e: No bootstrap required, opened a new log
I20260811 19:16:37.168380 667 tablet_bootstrap.cc:492] T 3e1ac3bfe05e4d3c92209c610486ad88 P 75742c8598794d92a22bdc0dfe029252: No bootstrap required, opened a new log
I20260811 19:16:37.168512 666 ts_tablet_manager.cc:1397] T 3e1ac3bfe05e4d3c92209c610486ad88 P 37701d3e508341709eb870f719d9d65e: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.007s
I20260811 19:16:37.168646 667 ts_tablet_manager.cc:1397] T 3e1ac3bfe05e4d3c92209c610486ad88 P 75742c8598794d92a22bdc0dfe029252: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260811 19:16:37.170842 669 tablet_bootstrap.cc:654] T 3e1ac3bfe05e4d3c92209c610486ad88 P a84233a301c3424fab33c25d4cfd771f: Neither blocks nor log segments found. Creating new log.
I20260811 19:16:37.172204 669 log.cc:824] T 3e1ac3bfe05e4d3c92209c610486ad88 P a84233a301c3424fab33c25d4cfd771f: Log is configured to *not* fsync() on all Append() calls
I20260811 19:16:37.176416 669 tablet_bootstrap.cc:492] T 3e1ac3bfe05e4d3c92209c610486ad88 P a84233a301c3424fab33c25d4cfd771f: No bootstrap required, opened a new log
I20260811 19:16:37.176800 669 ts_tablet_manager.cc:1397] T 3e1ac3bfe05e4d3c92209c610486ad88 P a84233a301c3424fab33c25d4cfd771f: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260811 19:16:37.177806 666 raft_consensus.cc:348] T 3e1ac3bfe05e4d3c92209c610486ad88 P 37701d3e508341709eb870f719d9d65e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a84233a301c3424fab33c25d4cfd771f" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 45715 } } peers { permanent_uuid: "75742c8598794d92a22bdc0dfe029252" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46815 } } peers { permanent_uuid: "37701d3e508341709eb870f719d9d65e" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41003 } }
I20260811 19:16:37.177771 667 raft_consensus.cc:348] T 3e1ac3bfe05e4d3c92209c610486ad88 P 75742c8598794d92a22bdc0dfe029252 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a84233a301c3424fab33c25d4cfd771f" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 45715 } } peers { permanent_uuid: "75742c8598794d92a22bdc0dfe029252" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46815 } } peers { permanent_uuid: "37701d3e508341709eb870f719d9d65e" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41003 } }
I20260811 19:16:37.178222 666 raft_consensus.cc:374] T 3e1ac3bfe05e4d3c92209c610486ad88 P 37701d3e508341709eb870f719d9d65e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:16:37.178257 667 raft_consensus.cc:374] T 3e1ac3bfe05e4d3c92209c610486ad88 P 75742c8598794d92a22bdc0dfe029252 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:16:37.178319 666 raft_consensus.cc:729] T 3e1ac3bfe05e4d3c92209c610486ad88 P 37701d3e508341709eb870f719d9d65e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37701d3e508341709eb870f719d9d65e, State: Initialized, Role: FOLLOWER
I20260811 19:16:37.178393 667 raft_consensus.cc:729] T 3e1ac3bfe05e4d3c92209c610486ad88 P 75742c8598794d92a22bdc0dfe029252 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75742c8598794d92a22bdc0dfe029252, State: Initialized, Role: FOLLOWER
I20260811 19:16:37.178972 666 consensus_queue.cc:260] T 3e1ac3bfe05e4d3c92209c610486ad88 P 37701d3e508341709eb870f719d9d65e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a84233a301c3424fab33c25d4cfd771f" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 45715 } } peers { permanent_uuid: "75742c8598794d92a22bdc0dfe029252" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46815 } } peers { permanent_uuid: "37701d3e508341709eb870f719d9d65e" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41003 } }
I20260811 19:16:37.179128 667 consensus_queue.cc:260] T 3e1ac3bfe05e4d3c92209c610486ad88 P 75742c8598794d92a22bdc0dfe029252 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a84233a301c3424fab33c25d4cfd771f" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 45715 } } peers { permanent_uuid: "75742c8598794d92a22bdc0dfe029252" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46815 } } peers { permanent_uuid: "37701d3e508341709eb870f719d9d65e" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41003 } }
I20260811 19:16:37.181144 667 ts_tablet_manager.cc:1428] T 3e1ac3bfe05e4d3c92209c610486ad88 P 75742c8598794d92a22bdc0dfe029252: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
I20260811 19:16:37.185717 669 raft_consensus.cc:348] T 3e1ac3bfe05e4d3c92209c610486ad88 P a84233a301c3424fab33c25d4cfd771f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a84233a301c3424fab33c25d4cfd771f" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 45715 } } peers { permanent_uuid: "75742c8598794d92a22bdc0dfe029252" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46815 } } peers { permanent_uuid: "37701d3e508341709eb870f719d9d65e" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41003 } }
I20260811 19:16:37.186131 669 raft_consensus.cc:374] T 3e1ac3bfe05e4d3c92209c610486ad88 P a84233a301c3424fab33c25d4cfd771f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:16:37.186250 669 raft_consensus.cc:729] T 3e1ac3bfe05e4d3c92209c610486ad88 P a84233a301c3424fab33c25d4cfd771f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a84233a301c3424fab33c25d4cfd771f, State: Initialized, Role: FOLLOWER
I20260811 19:16:37.186908 425 heartbeater.cc:503] Master 127.3.61.190:35205 was elected leader, sending a full tablet report...
I20260811 19:16:37.186911 669 consensus_queue.cc:260] T 3e1ac3bfe05e4d3c92209c610486ad88 P a84233a301c3424fab33c25d4cfd771f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a84233a301c3424fab33c25d4cfd771f" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 45715 } } peers { permanent_uuid: "75742c8598794d92a22bdc0dfe029252" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46815 } } peers { permanent_uuid: "37701d3e508341709eb870f719d9d65e" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41003 } }
I20260811 19:16:37.187467 666 ts_tablet_manager.cc:1428] T 3e1ac3bfe05e4d3c92209c610486ad88 P 37701d3e508341709eb870f719d9d65e: Time spent starting tablet: real 0.019s user 0.013s sys 0.006s
I20260811 19:16:37.189592 640 heartbeater.cc:503] Master 127.3.61.190:35205 was elected leader, sending a full tablet report...
I20260811 19:16:37.190389 669 ts_tablet_manager.cc:1428] T 3e1ac3bfe05e4d3c92209c610486ad88 P a84233a301c3424fab33c25d4cfd771f: Time spent starting tablet: real 0.013s user 0.015s sys 0.000s
I20260811 19:16:37.196380 674 raft_consensus.cc:482] T 3e1ac3bfe05e4d3c92209c610486ad88 P 37701d3e508341709eb870f719d9d65e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:16:37.196781 674 raft_consensus.cc:504] T 3e1ac3bfe05e4d3c92209c610486ad88 P 37701d3e508341709eb870f719d9d65e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a84233a301c3424fab33c25d4cfd771f" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 45715 } } peers { permanent_uuid: "75742c8598794d92a22bdc0dfe029252" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46815 } } peers { permanent_uuid: "37701d3e508341709eb870f719d9d65e" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41003 } }
I20260811 19:16:37.199008 674 leader_election.cc:288] T 3e1ac3bfe05e4d3c92209c610486ad88 P 37701d3e508341709eb870f719d9d65e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a84233a301c3424fab33c25d4cfd771f (127.3.61.131:45715), 75742c8598794d92a22bdc0dfe029252 (127.3.61.129:46815)
I20260811 19:16:37.206851 661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:37.199527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:52313 (local address 127.3.61.131:45715)
0811 19:16:37.199745 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:37.199752 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:37.200580 (+ 828us) server_negotiation.cc:407] Connection header received
0811 19:16:37.200876 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:37.200880 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:37.200946 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:37.201022 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:16:37.201596 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:37.202472 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:37.204548 (+ 2076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:37.205770 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:37.205969 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:37.206045 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:37.206442 (+ 397us) server_negotiation.cc:299] Negotiation successful
0811 19:16:37.206580 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 19:16:37.206938 678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:37.199297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.131:45715 (local address 127.3.61.130:52313)
0811 19:16:37.200440 (+ 1143us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:37.200466 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 19:16:37.200716 (+ 250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:16:37.201051 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:16:37.201056 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:16:37.201083 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:16:37.201457 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:37.201467 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:37.202604 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:37.202608 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:37.204360 (+ 1752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:37.204376 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:37.206018 (+ 1642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:37.206024 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:37.206200 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:37.206255 (+ 55us) client_negotiation.cc:769] Sending connection context
0811 19:16:37.206369 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 19:16:37.206518 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":943,"thread_start_us":747,"threads_started":1}
I20260811 19:16:37.206966 663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:37.199734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:43619 (local address 127.3.61.129:46815)
0811 19:16:37.199989 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:37.199995 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:37.200927 (+ 932us) server_negotiation.cc:407] Connection header received
0811 19:16:37.201071 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:37.201076 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:37.201150 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:37.201241 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:16:37.201945 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:37.202747 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:37.204758 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:37.206008 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:37.206168 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:37.206219 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:37.206593 (+ 374us) server_negotiation.cc:299] Negotiation successful
0811 19:16:37.206738 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 19:16:37.207010 681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:37.199600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:46815 (local address 127.3.61.130:43619)
0811 19:16:37.200767 (+ 1167us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:37.200784 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 19:16:37.200938 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:16:37.201285 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:16:37.201289 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:16:37.201321 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:16:37.201779 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:37.201794 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:37.202911 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:37.202915 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:37.204576 (+ 1661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:37.204593 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:37.206230 (+ 1637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:37.206236 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:37.206415 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:37.206445 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 19:16:37.206528 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 19:16:37.206638 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1048,"mutex_wait_us":7,"thread_start_us":836,"threads_started":1}
I20260811 19:16:37.208240 561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e1ac3bfe05e4d3c92209c610486ad88" candidate_uuid: "37701d3e508341709eb870f719d9d65e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a84233a301c3424fab33c25d4cfd771f" is_pre_election: true
I20260811 19:16:37.208240 32572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e1ac3bfe05e4d3c92209c610486ad88" candidate_uuid: "37701d3e508341709eb870f719d9d65e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75742c8598794d92a22bdc0dfe029252" is_pre_election: true
I20260811 19:16:37.208915 32572 raft_consensus.cc:2454] T 3e1ac3bfe05e4d3c92209c610486ad88 P 75742c8598794d92a22bdc0dfe029252 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 37701d3e508341709eb870f719d9d65e in term 0.
I20260811 19:16:37.208918 561 raft_consensus.cc:2454] T 3e1ac3bfe05e4d3c92209c610486ad88 P a84233a301c3424fab33c25d4cfd771f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 37701d3e508341709eb870f719d9d65e in term 0.
I20260811 19:16:37.209933 32703 leader_election.cc:302] T 3e1ac3bfe05e4d3c92209c610486ad88 P 37701d3e508341709eb870f719d9d65e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37701d3e508341709eb870f719d9d65e, a84233a301c3424fab33c25d4cfd771f; no voters:
I20260811 19:16:37.210388 674 raft_consensus.cc:2781] T 3e1ac3bfe05e4d3c92209c610486ad88 P 37701d3e508341709eb870f719d9d65e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:16:37.210559 674 raft_consensus.cc:482] T 3e1ac3bfe05e4d3c92209c610486ad88 P 37701d3e508341709eb870f719d9d65e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:16:37.210668 674 raft_consensus.cc:3037] T 3e1ac3bfe05e4d3c92209c610486ad88 P 37701d3e508341709eb870f719d9d65e [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:16:37.213367 674 raft_consensus.cc:504] T 3e1ac3bfe05e4d3c92209c610486ad88 P 37701d3e508341709eb870f719d9d65e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a84233a301c3424fab33c25d4cfd771f" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 45715 } } peers { permanent_uuid: "75742c8598794d92a22bdc0dfe029252" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46815 } } peers { permanent_uuid: "37701d3e508341709eb870f719d9d65e" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41003 } }
I20260811 19:16:37.214121 674 leader_election.cc:288] T 3e1ac3bfe05e4d3c92209c610486ad88 P 37701d3e508341709eb870f719d9d65e [CANDIDATE]: Term 1 election: Requested vote from peers a84233a301c3424fab33c25d4cfd771f (127.3.61.131:45715), 75742c8598794d92a22bdc0dfe029252 (127.3.61.129:46815)
I20260811 19:16:37.214694 561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e1ac3bfe05e4d3c92209c610486ad88" candidate_uuid: "37701d3e508341709eb870f719d9d65e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a84233a301c3424fab33c25d4cfd771f"
I20260811 19:16:37.214743 32572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e1ac3bfe05e4d3c92209c610486ad88" candidate_uuid: "37701d3e508341709eb870f719d9d65e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75742c8598794d92a22bdc0dfe029252"
I20260811 19:16:37.214915 561 raft_consensus.cc:3037] T 3e1ac3bfe05e4d3c92209c610486ad88 P a84233a301c3424fab33c25d4cfd771f [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:16:37.215116 32572 raft_consensus.cc:3037] T 3e1ac3bfe05e4d3c92209c610486ad88 P 75742c8598794d92a22bdc0dfe029252 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:16:37.217351 561 raft_consensus.cc:2454] T 3e1ac3bfe05e4d3c92209c610486ad88 P a84233a301c3424fab33c25d4cfd771f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 37701d3e508341709eb870f719d9d65e in term 1.
I20260811 19:16:37.217908 32703 leader_election.cc:302] T 3e1ac3bfe05e4d3c92209c610486ad88 P 37701d3e508341709eb870f719d9d65e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37701d3e508341709eb870f719d9d65e, a84233a301c3424fab33c25d4cfd771f; no voters:
I20260811 19:16:37.218214 674 raft_consensus.cc:2781] T 3e1ac3bfe05e4d3c92209c610486ad88 P 37701d3e508341709eb870f719d9d65e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:16:37.218438 32572 raft_consensus.cc:2454] T 3e1ac3bfe05e4d3c92209c610486ad88 P 75742c8598794d92a22bdc0dfe029252 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 37701d3e508341709eb870f719d9d65e in term 1.
I20260811 19:16:37.218797 674 raft_consensus.cc:686] T 3e1ac3bfe05e4d3c92209c610486ad88 P 37701d3e508341709eb870f719d9d65e [term 1 LEADER]: Becoming Leader. State: Replica: 37701d3e508341709eb870f719d9d65e, State: Running, Role: LEADER
I20260811 19:16:37.219437 674 consensus_queue.cc:237] T 3e1ac3bfe05e4d3c92209c610486ad88 P 37701d3e508341709eb870f719d9d65e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a84233a301c3424fab33c25d4cfd771f" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 45715 } } peers { permanent_uuid: "75742c8598794d92a22bdc0dfe029252" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46815 } } peers { permanent_uuid: "37701d3e508341709eb870f719d9d65e" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41003 } }
I20260811 19:16:37.224571 32317 catalog_manager.cc:5495] T 3e1ac3bfe05e4d3c92209c610486ad88 P 37701d3e508341709eb870f719d9d65e reported cstate change: term changed from 0 to 1, leader changed from <none> to 37701d3e508341709eb870f719d9d65e (127.3.61.130). New cstate: current_term: 1 leader_uuid: "37701d3e508341709eb870f719d9d65e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a84233a301c3424fab33c25d4cfd771f" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 45715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75742c8598794d92a22bdc0dfe029252" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "37701d3e508341709eb870f719d9d65e" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41003 } health_report { overall_health: HEALTHY } } }
W20260811 19:16:37.264617 32657 tablet.cc:2369] T 3e1ac3bfe05e4d3c92209c610486ad88 P 75742c8598794d92a22bdc0dfe029252: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:16:37.266320 665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:37.259277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56290 (local address 127.3.61.130:41003)
0811 19:16:37.259603 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:37.259609 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:37.259634 (+ 25us) server_negotiation.cc:407] Connection header received
0811 19:16:37.259772 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:37.259777 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:37.259885 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:37.260067 (+ 182us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:16:37.260647 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:37.261438 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:37.263038 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:37.263482 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:37.263590 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:37.264656 (+ 1066us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:16:37.265519 (+ 863us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:16:37.265741 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:37.265883 (+ 142us) server_negotiation.cc:299] Negotiation successful
0811 19:16:37.265992 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260811 19:16:37.275490 32572 raft_consensus.cc:1261] T 3e1ac3bfe05e4d3c92209c610486ad88 P 75742c8598794d92a22bdc0dfe029252 [term 1 FOLLOWER]: Refusing update from remote peer 37701d3e508341709eb870f719d9d65e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:16:37.275530 561 raft_consensus.cc:1261] T 3e1ac3bfe05e4d3c92209c610486ad88 P a84233a301c3424fab33c25d4cfd771f [term 1 FOLLOWER]: Refusing update from remote peer 37701d3e508341709eb870f719d9d65e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:16:37.276556 682 consensus_queue.cc:1035] T 3e1ac3bfe05e4d3c92209c610486ad88 P 37701d3e508341709eb870f719d9d65e [LEADER]: Connected to new peer: Peer: permanent_uuid: "75742c8598794d92a22bdc0dfe029252" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 46815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:16:37.277016 674 consensus_queue.cc:1035] T 3e1ac3bfe05e4d3c92209c610486ad88 P 37701d3e508341709eb870f719d9d65e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a84233a301c3424fab33c25d4cfd771f" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 45715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 19:16:37.287746 642 tablet.cc:2369] T 3e1ac3bfe05e4d3c92209c610486ad88 P a84233a301c3424fab33c25d4cfd771f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:16:37.291260 692 mvcc.cc:205] Tried to move back new op lower bound from 7317404865629388800 to 7317404865415229440. Current Snapshot: MvccSnapshot[applied={T|T < 7317404865629388800}]
I20260811 19:16:37.305806 32317 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38106:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:16:37.306138 32317 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38106:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:16:37.310406 32317 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 13997ebd774f4dde9bde004c768ea3eb: Sending DeleteTablet for 3 replicas of tablet 3e1ac3bfe05e4d3c92209c610486ad88
I20260811 19:16:37.311928 534 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e1ac3bfe05e4d3c92209c610486ad88 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:16:37 UTC) from {username='kudu', principal='kudu/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:58552
I20260811 19:16:37.311926 32541 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e1ac3bfe05e4d3c92209c610486ad88 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:16:37 UTC) from {username='kudu', principal='kudu/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:51780
I20260811 19:16:37.313071 325 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e1ac3bfe05e4d3c92209c610486ad88 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:16:37 UTC) from {username='kudu', principal='kudu/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:56288
I20260811 19:16:37.314301 708 tablet_replica.cc:330] T 3e1ac3bfe05e4d3c92209c610486ad88 P 37701d3e508341709eb870f719d9d65e: stopping tablet replica
I20260811 19:16:37.314391 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 32428
I20260811 19:16:37.314891 708 raft_consensus.cc:2229] T 3e1ac3bfe05e4d3c92209c610486ad88 P 37701d3e508341709eb870f719d9d65e [term 1 LEADER]: Raft consensus shutting down.
I20260811 19:16:37.315649 708 raft_consensus.cc:2258] T 3e1ac3bfe05e4d3c92209c610486ad88 P 37701d3e508341709eb870f719d9d65e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:16:37.318922 708 ts_tablet_manager.cc:1905] T 3e1ac3bfe05e4d3c92209c610486ad88 P 37701d3e508341709eb870f719d9d65e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:16:37.324961 701 tablet_replica.cc:330] T 3e1ac3bfe05e4d3c92209c610486ad88 P 75742c8598794d92a22bdc0dfe029252: stopping tablet replica
I20260811 19:16:37.325675 706 tablet_replica.cc:330] T 3e1ac3bfe05e4d3c92209c610486ad88 P a84233a301c3424fab33c25d4cfd771f: stopping tablet replica
I20260811 19:16:37.326421 706 raft_consensus.cc:2229] T 3e1ac3bfe05e4d3c92209c610486ad88 P a84233a301c3424fab33c25d4cfd771f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:16:37.326929 708 ts_tablet_manager.cc:1918] T 3e1ac3bfe05e4d3c92209c610486ad88 P 37701d3e508341709eb870f719d9d65e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:16:37.326987 701 raft_consensus.cc:2229] T 3e1ac3bfe05e4d3c92209c610486ad88 P 75742c8598794d92a22bdc0dfe029252 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:16:37.327643 701 raft_consensus.cc:2258] T 3e1ac3bfe05e4d3c92209c610486ad88 P 75742c8598794d92a22bdc0dfe029252 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:16:37.327927 706 raft_consensus.cc:2258] T 3e1ac3bfe05e4d3c92209c610486ad88 P a84233a301c3424fab33c25d4cfd771f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:16:37.330190 701 ts_tablet_manager.cc:1905] T 3e1ac3bfe05e4d3c92209c610486ad88 P 75742c8598794d92a22bdc0dfe029252: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:16:37.330191 706 ts_tablet_manager.cc:1905] T 3e1ac3bfe05e4d3c92209c610486ad88 P a84233a301c3424fab33c25d4cfd771f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:16:37.335762 708 log.cc:1196] T 3e1ac3bfe05e4d3c92209c610486ad88 P 37701d3e508341709eb870f719d9d65e: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/ts-1/wal/wals/3e1ac3bfe05e4d3c92209c610486ad88
I20260811 19:16:37.338302 706 ts_tablet_manager.cc:1918] T 3e1ac3bfe05e4d3c92209c610486ad88 P a84233a301c3424fab33c25d4cfd771f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:16:37.338302 701 ts_tablet_manager.cc:1918] T 3e1ac3bfe05e4d3c92209c610486ad88 P 75742c8598794d92a22bdc0dfe029252: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:16:37.338639 701 log.cc:1196] T 3e1ac3bfe05e4d3c92209c610486ad88 P 75742c8598794d92a22bdc0dfe029252: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/ts-0/wal/wals/3e1ac3bfe05e4d3c92209c610486ad88
I20260811 19:16:37.338666 706 log.cc:1196] T 3e1ac3bfe05e4d3c92209c610486ad88 P a84233a301c3424fab33c25d4cfd771f: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786475454804547-3318-0/minicluster-data/ts-2/wal/wals/3e1ac3bfe05e4d3c92209c610486ad88
I20260811 19:16:37.339298 701 ts_tablet_manager.cc:1939] T 3e1ac3bfe05e4d3c92209c610486ad88 P 75742c8598794d92a22bdc0dfe029252: Deleting consensus metadata
I20260811 19:16:37.339354 706 ts_tablet_manager.cc:1939] T 3e1ac3bfe05e4d3c92209c610486ad88 P a84233a301c3424fab33c25d4cfd771f: Deleting consensus metadata
Aug 11 19:16:37 dist-test-slave-t7g5 krb5kdc[32245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475794, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.129@KRBTEST.COM
I20260811 19:16:37.340296 708 ts_tablet_manager.cc:1939] T 3e1ac3bfe05e4d3c92209c610486ad88 P 37701d3e508341709eb870f719d9d65e: Deleting consensus metadata
I20260811 19:16:37.342278 32292 catalog_manager.cc:4841] TS 37701d3e508341709eb870f719d9d65e (127.3.61.130:41003): tablet 3e1ac3bfe05e4d3c92209c610486ad88 (table test-table [id=1d170f49fb9a4cb582994d10c979ef5a]) successfully deleted
I20260811 19:16:37.342877 32292 catalog_manager.cc:4841] TS a84233a301c3424fab33c25d4cfd771f (127.3.61.131:45715): tablet 3e1ac3bfe05e4d3c92209c610486ad88 (table test-table [id=1d170f49fb9a4cb582994d10c979ef5a]) successfully deleted
I20260811 19:16:37.342163 32291 catalog_manager.cc:4841] TS 75742c8598794d92a22bdc0dfe029252 (127.3.61.129:46815): tablet 3e1ac3bfe05e4d3c92209c610486ad88 (table test-table [id=1d170f49fb9a4cb582994d10c979ef5a]) successfully deleted
I20260811 19:16:37.350275 663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:37.317061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51788 (local address 127.3.61.129:46815)
0811 19:16:37.317265 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:37.317273 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:37.327678 (+ 10405us) server_negotiation.cc:407] Connection header received
0811 19:16:37.328016 (+ 338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:37.328022 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:37.328121 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:37.328201 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:16:37.330668 (+ 2467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:37.331869 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:37.333956 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:37.334647 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:37.334756 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:37.343543 (+ 8787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:16:37.343559 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:16:37.343574 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:16:37.343615 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:16:37.346910 (+ 3295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:16:37.347955 (+ 1045us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:16:37.347963 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:16:37.347966 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:16:37.348121 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:16:37.348607 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:16:37.348612 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:16:37.348614 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:16:37.349030 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:16:37.349294 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:37.349765 (+ 471us) server_negotiation.cc:299] Negotiation successful
0811 19:16:37.349963 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 19:16:37.350971 32646 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:16:37.478875 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 32428
I20260811 19:16:37.514300 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 32670
Aug 11 19:16:37 dist-test-slave-t7g5 krb5kdc[32245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475794, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.130@KRBTEST.COM
I20260811 19:16:37.532419 665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:37.515194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56296 (local address 127.3.61.130:41003)
0811 19:16:37.515403 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:37.515408 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:37.515442 (+ 34us) server_negotiation.cc:407] Connection header received
0811 19:16:37.515573 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:37.515579 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:37.515654 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:37.515747 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:16:37.517127 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:37.518557 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:37.520327 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:37.520874 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:37.521111 (+ 237us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:37.526038 (+ 4927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:16:37.526058 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:16:37.526076 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:16:37.526130 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:16:37.529214 (+ 3084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:16:37.530199 (+ 985us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:16:37.530207 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:16:37.530209 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:16:37.530386 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:16:37.530795 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:16:37.530799 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:16:37.530801 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:16:37.531192 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:16:37.531539 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:37.531829 (+ 290us) server_negotiation.cc:299] Negotiation successful
0811 19:16:37.532062 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 19:16:37.533160 419 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:16:37.655186 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 32670
I20260811 19:16:37.678320 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 434
Aug 11 19:16:37 dist-test-slave-t7g5 krb5kdc[32245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475794, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.131@KRBTEST.COM
I20260811 19:16:37.696098 661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:37.679186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58562 (local address 127.3.61.131:45715)
0811 19:16:37.679380 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:37.679385 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:37.679405 (+ 20us) server_negotiation.cc:407] Connection header received
0811 19:16:37.679513 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:37.679518 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:37.679586 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:37.679678 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:16:37.681773 (+ 2095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:37.682577 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:37.684135 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:37.684596 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:37.684721 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:37.689331 (+ 4610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:16:37.689350 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:16:37.689368 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:16:37.689416 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:16:37.692300 (+ 2884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:16:37.693188 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:16:37.693202 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:16:37.693205 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:16:37.693470 (+ 265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:16:37.694117 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:16:37.694122 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:16:37.694125 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:16:37.694699 (+ 574us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:16:37.695016 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:37.695566 (+ 550us) server_negotiation.cc:299] Negotiation successful
0811 19:16:37.695785 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 19:16:37.696853 631 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:16:37.791597 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 434
I20260811 19:16:37.823608 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 32270
I20260811 19:16:37.825022 32378 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:16:37.942302 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 32270
2026-08-11T19:16:37Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (5176 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-11T19:16:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:16:38Z Disabled control of system clock
I20260811 19:16:38.062985 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:38067
--webserver_interface=127.3.61.190
--webserver_port=0
--builtin_ntp_servers=127.3.61.148:40523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.61.190:38067
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 19:16:38.472437 725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:16:38.472792 725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:16:38.472937 725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:16:38.482301 725 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:16:38.482455 725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:16:38.482519 725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:16:38.482601 725 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:16:38.482663 725 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:16:38.496668 725 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:40523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.61.190:38067
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:38067
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.61.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:16:38.498236 725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:16:38.500556 725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:16:38.517232 732 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:16:38.517345 733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:16:38.518076 736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:16:38.519920 725 server_base.cc:1034] running on GCE node
I20260811 19:16:38.521207 725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:16:38.523159 725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:16:38.524451 725 hybrid_clock.cc:648] HybridClock initialized: now 1786475798524400 us; error 60 us; skew 500 ppm
I20260811 19:16:38.524897 725 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 19:16:38.528825 725 webserver.cc:460] Webserver started at http://127.3.61.190:42953/ using document root <none> and password file <none>
I20260811 19:16:38.529901 725 fs_manager.cc:359] Metadata directory not provided
I20260811 19:16:38.530040 725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:16:38.530411 725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:16:38.534289 725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/master-0/data/instance:
uuid: "fa27dd66f1c741b1ae5199df9ca13e6a"
format_stamp: "Formatted at 2026-08-11 19:16:38 on dist-test-slave-t7g5"
server_key: "06e69c86fbfca633d2e33398d49942b6"
server_key_iv: "fdbfdc2bf33e4dfca3035124818b1913"
server_key_version: "encryptionkey@0"
I20260811 19:16:38.535405 725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/master-0/wal/instance:
uuid: "fa27dd66f1c741b1ae5199df9ca13e6a"
format_stamp: "Formatted at 2026-08-11 19:16:38 on dist-test-slave-t7g5"
server_key: "06e69c86fbfca633d2e33398d49942b6"
server_key_iv: "fdbfdc2bf33e4dfca3035124818b1913"
server_key_version: "encryptionkey@0"
I20260811 19:16:38.541433 725 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260811 19:16:38.546962 747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:38.548727 725 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260811 19:16:38.548938 725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/master-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/master-0/wal
uuid: "fa27dd66f1c741b1ae5199df9ca13e6a"
format_stamp: "Formatted at 2026-08-11 19:16:38 on dist-test-slave-t7g5"
server_key: "06e69c86fbfca633d2e33398d49942b6"
server_key_iv: "fdbfdc2bf33e4dfca3035124818b1913"
server_key_version: "encryptionkey@0"
I20260811 19:16:38.549167 725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:16:38.587342 725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:16:38.588531 725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:16:38.588855 725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:16:38.610244 725 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.190:38067
I20260811 19:16:38.610273 840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.190:38067 every 8 connection(s)
I20260811 19:16:38.611667 725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
I20260811 19:16:38.614547 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 725
I20260811 19:16:38.614751 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/master-0/wal/instance
I20260811 19:16:38.615339 3318 external_mini_cluster.cc:1428] Setting key 2cccb6acd1d68c19f8c919b2feb3689c
I20260811 19:16:38.616772 841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:16:38.630702 849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:38.617449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60588 (local address 127.3.61.190:38067)
0811 19:16:38.618503 (+ 1054us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:38.618514 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:38.618561 (+ 47us) server_negotiation.cc:407] Connection header received
0811 19:16:38.620176 (+ 1615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:38.620198 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:38.620807 (+ 609us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:38.621164 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:16:38.621866 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:38.623220 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:38.625365 (+ 2145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:38.626161 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:38.626474 (+ 313us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:38.628060 (+ 1586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:16:38.628069 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:16:38.628086 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:16:38.628679 (+ 593us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:16:38.628773 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:16:38.628798 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:16:38.628898 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:38.629197 (+ 299us) server_negotiation.cc:299] Negotiation successful
0811 19:16:38.629588 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":635,"thread_start_us":411,"threads_started":1}
I20260811 19:16:38.632193 841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa27dd66f1c741b1ae5199df9ca13e6a: Bootstrap starting.
I20260811 19:16:38.635618 841 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fa27dd66f1c741b1ae5199df9ca13e6a: Neither blocks nor log segments found. Creating new log.
I20260811 19:16:38.636866 841 log.cc:824] T 00000000000000000000000000000000 P fa27dd66f1c741b1ae5199df9ca13e6a: Log is configured to *not* fsync() on all Append() calls
I20260811 19:16:38.640389 841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa27dd66f1c741b1ae5199df9ca13e6a: No bootstrap required, opened a new log
I20260811 19:16:38.649142 841 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa27dd66f1c741b1ae5199df9ca13e6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa27dd66f1c741b1ae5199df9ca13e6a" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 38067 } }
I20260811 19:16:38.649614 841 raft_consensus.cc:374] T 00000000000000000000000000000000 P fa27dd66f1c741b1ae5199df9ca13e6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:16:38.649750 841 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa27dd66f1c741b1ae5199df9ca13e6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa27dd66f1c741b1ae5199df9ca13e6a, State: Initialized, Role: FOLLOWER
I20260811 19:16:38.650653 841 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa27dd66f1c741b1ae5199df9ca13e6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa27dd66f1c741b1ae5199df9ca13e6a" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 38067 } }
I20260811 19:16:38.651125 841 raft_consensus.cc:388] T 00000000000000000000000000000000 P fa27dd66f1c741b1ae5199df9ca13e6a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:16:38.651312 841 raft_consensus.cc:482] T 00000000000000000000000000000000 P fa27dd66f1c741b1ae5199df9ca13e6a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:16:38.651505 841 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa27dd66f1c741b1ae5199df9ca13e6a [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:16:38.654264 841 raft_consensus.cc:504] T 00000000000000000000000000000000 P fa27dd66f1c741b1ae5199df9ca13e6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa27dd66f1c741b1ae5199df9ca13e6a" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 38067 } }
I20260811 19:16:38.654892 841 leader_election.cc:302] T 00000000000000000000000000000000 P fa27dd66f1c741b1ae5199df9ca13e6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fa27dd66f1c741b1ae5199df9ca13e6a; no voters:
I20260811 19:16:38.655478 841 leader_election.cc:288] T 00000000000000000000000000000000 P fa27dd66f1c741b1ae5199df9ca13e6a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:16:38.655619 851 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fa27dd66f1c741b1ae5199df9ca13e6a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:16:38.656332 851 raft_consensus.cc:686] T 00000000000000000000000000000000 P fa27dd66f1c741b1ae5199df9ca13e6a [term 1 LEADER]: Becoming Leader. State: Replica: fa27dd66f1c741b1ae5199df9ca13e6a, State: Running, Role: LEADER
I20260811 19:16:38.656872 851 consensus_queue.cc:237] T 00000000000000000000000000000000 P fa27dd66f1c741b1ae5199df9ca13e6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa27dd66f1c741b1ae5199df9ca13e6a" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 38067 } }
I20260811 19:16:38.657555 841 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa27dd66f1c741b1ae5199df9ca13e6a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:16:38.660678 861 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa27dd66f1c741b1ae5199df9ca13e6a [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa27dd66f1c741b1ae5199df9ca13e6a. Latest consensus state: current_term: 1 leader_uuid: "fa27dd66f1c741b1ae5199df9ca13e6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa27dd66f1c741b1ae5199df9ca13e6a" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 38067 } } }
I20260811 19:16:38.661058 859 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa27dd66f1c741b1ae5199df9ca13e6a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fa27dd66f1c741b1ae5199df9ca13e6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa27dd66f1c741b1ae5199df9ca13e6a" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 38067 } } }
I20260811 19:16:38.662650 861 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa27dd66f1c741b1ae5199df9ca13e6a [sys.catalog]: This master's current role is: LEADER
I20260811 19:16:38.662753 859 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa27dd66f1c741b1ae5199df9ca13e6a [sys.catalog]: This master's current role is: LEADER
I20260811 19:16:38.663762 878 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:16:38.673594 878 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:16:38.685420 878 catalog_manager.cc:1347] Generated new cluster ID: 56ba3f0deae24661b65d85ed9f4aca88
I20260811 19:16:38.685621 878 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:16:38.700625 878 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:16:38.702545 878 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:16:38.713902 878 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fa27dd66f1c741b1ae5199df9ca13e6a: Generated new TSK 0
I20260811 19:16:38.714838 878 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:16:38.720043 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.129:0
--local_ip_for_outbound_sockets=127.3.61.129
--webserver_interface=127.3.61.129
--webserver_port=0
--tserver_master_addrs=127.3.61.190:38067
--builtin_ntp_servers=127.3.61.148:40523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 19:16:39.118608 893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:16:39.118947 893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:16:39.119059 893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:16:39.127071 893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:16:39.127279 893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:16:39.127462 893 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.129
I20260811 19:16:39.137761 893 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:40523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.61.129
--webserver_port=0
--tserver_master_addrs=127.3.61.190:38067
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.129
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:16:39.139320 893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:16:39.141268 893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:16:39.156699 905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:16:39.156765 901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:16:39.156728 902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:16:39.159229 893 server_base.cc:1034] running on GCE node
I20260811 19:16:39.160027 893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:16:39.161314 893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:16:39.162715 893 hybrid_clock.cc:648] HybridClock initialized: now 1786475799162626 us; error 110 us; skew 500 ppm
I20260811 19:16:39.163129 893 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 19:16:39.165925 893 webserver.cc:460] Webserver started at http://127.3.61.129:40739/ using document root <none> and password file <none>
I20260811 19:16:39.166697 893 fs_manager.cc:359] Metadata directory not provided
I20260811 19:16:39.166826 893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:16:39.167167 893 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:16:39.170373 893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/ts-0/data/instance:
uuid: "3d14526964684cf6b179a72a679734c2"
format_stamp: "Formatted at 2026-08-11 19:16:39 on dist-test-slave-t7g5"
server_key: "949e57279be136a58f8a8f4109f64944"
server_key_iv: "4b0760fa72cc711b0baffe851e0014cd"
server_key_version: "encryptionkey@0"
I20260811 19:16:39.171236 893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/ts-0/wal/instance:
uuid: "3d14526964684cf6b179a72a679734c2"
format_stamp: "Formatted at 2026-08-11 19:16:39 on dist-test-slave-t7g5"
server_key: "949e57279be136a58f8a8f4109f64944"
server_key_iv: "4b0760fa72cc711b0baffe851e0014cd"
server_key_version: "encryptionkey@0"
I20260811 19:16:39.177615 893 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260811 19:16:39.183032 918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:39.184877 893 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260811 19:16:39.185122 893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/ts-0/wal
uuid: "3d14526964684cf6b179a72a679734c2"
format_stamp: "Formatted at 2026-08-11 19:16:39 on dist-test-slave-t7g5"
server_key: "949e57279be136a58f8a8f4109f64944"
server_key_iv: "4b0760fa72cc711b0baffe851e0014cd"
server_key_version: "encryptionkey@0"
I20260811 19:16:39.185422 893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:16:39.219429 893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:16:39.220762 893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:16:39.221190 893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:16:39.222440 893 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:16:39.224087 893 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:16:39.224195 893 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:39.224313 893 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:16:39.224390 893 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:39.258910 893 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.129:38637
I20260811 19:16:39.258975 1092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.129:38637 every 8 connection(s)
I20260811 19:16:39.260296 893 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
I20260811 19:16:39.270315 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 893
I20260811 19:16:39.270529 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/ts-0/wal/instance
I20260811 19:16:39.271116 3318 external_mini_cluster.cc:1428] Setting key beb47d0db1cb1c8fa5a0a56b23dc636e
I20260811 19:16:39.274791 1097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:39.262999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:38571 (local address 127.3.61.190:38067)
0811 19:16:39.263723 (+ 724us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:39.263730 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:39.264931 (+ 1201us) server_negotiation.cc:407] Connection header received
0811 19:16:39.266934 (+ 2003us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:39.266941 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:39.267042 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:39.267138 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:16:39.268319 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:39.269076 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:39.271301 (+ 2225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:39.271727 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:39.271830 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:39.273740 (+ 1910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:16:39.273746 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:16:39.273750 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:16:39.273771 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:16:39.273819 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:16:39.273836 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:16:39.273936 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:39.274270 (+ 334us) server_negotiation.cc:299] Negotiation successful
0811 19:16:39.274406 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":529,"thread_start_us":342,"threads_started":1}
I20260811 19:16:39.275417 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.130:0
--local_ip_for_outbound_sockets=127.3.61.130
--webserver_interface=127.3.61.130
--webserver_port=0
--tserver_master_addrs=127.3.61.190:38067
--builtin_ntp_servers=127.3.61.148:40523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 19:16:39.276130 1098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:39.263628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:38067 (local address 127.3.61.129:38571)
0811 19:16:39.264737 (+ 1109us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:39.264807 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0811 19:16:39.266626 (+ 1819us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:16:39.267419 (+ 793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:16:39.267428 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:16:39.267524 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:16:39.268142 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:39.268160 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:39.269236 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:39.269240 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:39.271052 (+ 1812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:39.271076 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:39.272021 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:39.272031 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:39.272313 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:39.273249 (+ 936us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 19:16:39.273252 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:16:39.273323 (+ 71us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 19:16:39.273441 (+ 118us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:16:39.273977 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:16:39.273991 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:16:39.273999 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 19:16:39.274217 (+ 218us) client_negotiation.cc:240] Negotiation successful
0811 19:16:39.274596 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":314,"threads_started":1}
I20260811 19:16:39.277997 1094 heartbeater.cc:347] Connected to a master server at 127.3.61.190:38067
I20260811 19:16:39.278367 1094 heartbeater.cc:464] Registering TS with master...
I20260811 19:16:39.279286 1094 heartbeater.cc:511] Master 127.3.61.190:38067 requested a full tablet report, sending...
I20260811 19:16:39.281622 771 ts_manager.cc:194] Registered new tserver with Master: 3d14526964684cf6b179a72a679734c2 (127.3.61.129:38637)
I20260811 19:16:39.283169 771 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.61.129:38571
W20260811 19:16:39.645378 1102 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:16:39.645714 1102 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:16:39.645855 1102 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:16:39.653309 1102 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:16:39.653435 1102 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:16:39.653579 1102 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.130
I20260811 19:16:39.663134 1102 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:40523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.61.130
--webserver_port=0
--tserver_master_addrs=127.3.61.190:38067
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.130
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:16:39.664511 1102 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:16:39.666452 1102 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:16:39.680418 1109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:16:39.680418 1113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:16:39.680356 1108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:16:39.681419 1102 server_base.cc:1034] running on GCE node
I20260811 19:16:39.682117 1102 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:16:39.683293 1102 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:16:39.684546 1102 hybrid_clock.cc:648] HybridClock initialized: now 1786475799684500 us; error 38 us; skew 500 ppm
I20260811 19:16:39.684859 1102 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 19:16:39.687084 1102 webserver.cc:460] Webserver started at http://127.3.61.130:45357/ using document root <none> and password file <none>
I20260811 19:16:39.687800 1102 fs_manager.cc:359] Metadata directory not provided
I20260811 19:16:39.687918 1102 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:16:39.688247 1102 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:16:39.691690 1102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/ts-1/data/instance:
uuid: "2a96f8f035e349fdb3963bf6917a72f2"
format_stamp: "Formatted at 2026-08-11 19:16:39 on dist-test-slave-t7g5"
server_key: "e184ea037c97af92ec4edda3a37302c5"
server_key_iv: "0f6eafda92ce4867aba4f2de7acd7037"
server_key_version: "encryptionkey@0"
I20260811 19:16:39.692512 1102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/ts-1/wal/instance:
uuid: "2a96f8f035e349fdb3963bf6917a72f2"
format_stamp: "Formatted at 2026-08-11 19:16:39 on dist-test-slave-t7g5"
server_key: "e184ea037c97af92ec4edda3a37302c5"
server_key_iv: "0f6eafda92ce4867aba4f2de7acd7037"
server_key_version: "encryptionkey@0"
I20260811 19:16:39.698019 1102 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260811 19:16:39.701768 1121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:39.703125 1102 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 19:16:39.703284 1102 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/ts-1/wal
uuid: "2a96f8f035e349fdb3963bf6917a72f2"
format_stamp: "Formatted at 2026-08-11 19:16:39 on dist-test-slave-t7g5"
server_key: "e184ea037c97af92ec4edda3a37302c5"
server_key_iv: "0f6eafda92ce4867aba4f2de7acd7037"
server_key_version: "encryptionkey@0"
I20260811 19:16:39.703598 1102 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:16:39.731987 1102 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:16:39.732873 1102 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:16:39.733166 1102 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:16:39.734225 1102 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:16:39.735900 1102 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:16:39.735996 1102 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:39.736107 1102 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:16:39.736183 1102 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:39.772164 1102 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.130:41901
I20260811 19:16:39.772258 1310 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.130:41901 every 8 connection(s)
I20260811 19:16:39.773476 1102 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
I20260811 19:16:39.783128 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 1102
I20260811 19:16:39.783295 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/ts-1/wal/instance
I20260811 19:16:39.783722 3318 external_mini_cluster.cc:1428] Setting key cbaec02956bd85b8c664f789895928ef
I20260811 19:16:39.787180 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.131:0
--local_ip_for_outbound_sockets=127.3.61.131
--webserver_interface=127.3.61.131
--webserver_port=0
--tserver_master_addrs=127.3.61.190:38067
--builtin_ntp_servers=127.3.61.148:40523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 19:16:39.787477 1313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:39.776995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:43537 (local address 127.3.61.190:38067)
0811 19:16:39.777455 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:39.777460 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:39.778762 (+ 1302us) server_negotiation.cc:407] Connection header received
0811 19:16:39.780623 (+ 1861us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:39.780629 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:39.780710 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:39.780804 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:16:39.781949 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:39.782680 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:39.784393 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:39.784944 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:39.785076 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:39.786600 (+ 1524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:16:39.786605 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:16:39.786608 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:16:39.786626 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:16:39.786668 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:16:39.786680 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:16:39.786756 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:39.787045 (+ 289us) server_negotiation.cc:299] Negotiation successful
0811 19:16:39.787129 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":227,"threads_started":1}
I20260811 19:16:39.788758 1315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:39.777545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:38067 (local address 127.3.61.130:43537)
0811 19:16:39.778573 (+ 1028us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:39.778618 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 19:16:39.780356 (+ 1738us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:16:39.781056 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:16:39.781065 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:16:39.781150 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:16:39.781780 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:39.781800 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:39.782841 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:39.782846 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:39.784206 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:39.784223 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:39.785181 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:39.785187 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:39.785388 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:39.786223 (+ 835us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 19:16:39.786226 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:16:39.786288 (+ 62us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 19:16:39.786392 (+ 104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:16:39.786781 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:16:39.786793 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:16:39.786801 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 19:16:39.787013 (+ 212us) client_negotiation.cc:240] Negotiation successful
0811 19:16:39.787437 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":301,"threads_started":1}
I20260811 19:16:39.790601 1311 heartbeater.cc:347] Connected to a master server at 127.3.61.190:38067
I20260811 19:16:39.790946 1311 heartbeater.cc:464] Registering TS with master...
I20260811 19:16:39.791865 1311 heartbeater.cc:511] Master 127.3.61.190:38067 requested a full tablet report, sending...
I20260811 19:16:39.793331 771 ts_manager.cc:194] Registered new tserver with Master: 2a96f8f035e349fdb3963bf6917a72f2 (127.3.61.130:41901)
I20260811 19:16:39.794158 771 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.61.130:43537
W20260811 19:16:40.155416 1319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:16:40.155738 1319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:16:40.155841 1319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:16:40.163815 1319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:16:40.163939 1319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:16:40.164074 1319 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.131
I20260811 19:16:40.174103 1319 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:40523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.61.131
--webserver_port=0
--tserver_master_addrs=127.3.61.190:38067
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.131
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:16:40.175486 1319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:16:40.177350 1319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:16:40.191148 1330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:16:40.191172 1334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:16:40.191896 1331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:16:40.192900 1319 server_base.cc:1034] running on GCE node
I20260811 19:16:40.193738 1319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:16:40.195052 1319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:16:40.196382 1319 hybrid_clock.cc:648] HybridClock initialized: now 1786475800196278 us; error 49 us; skew 500 ppm
I20260811 19:16:40.196996 1319 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 19:16:40.203476 1319 webserver.cc:460] Webserver started at http://127.3.61.131:39229/ using document root <none> and password file <none>
I20260811 19:16:40.204236 1319 fs_manager.cc:359] Metadata directory not provided
I20260811 19:16:40.204346 1319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:16:40.204675 1319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:16:40.208395 1319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/ts-2/data/instance:
uuid: "a0481b8231954a0ea2dde6d3dece7cce"
format_stamp: "Formatted at 2026-08-11 19:16:40 on dist-test-slave-t7g5"
server_key: "42de65de52b6ee2d03d2769b95614370"
server_key_iv: "945bde7eb31b4eaf7834deb1e5cf97b1"
server_key_version: "encryptionkey@0"
I20260811 19:16:40.209744 1319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/ts-2/wal/instance:
uuid: "a0481b8231954a0ea2dde6d3dece7cce"
format_stamp: "Formatted at 2026-08-11 19:16:40 on dist-test-slave-t7g5"
server_key: "42de65de52b6ee2d03d2769b95614370"
server_key_iv: "945bde7eb31b4eaf7834deb1e5cf97b1"
server_key_version: "encryptionkey@0"
I20260811 19:16:40.215845 1319 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 19:16:40.219887 1342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:40.221606 1319 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 19:16:40.221832 1319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/ts-2/wal
uuid: "a0481b8231954a0ea2dde6d3dece7cce"
format_stamp: "Formatted at 2026-08-11 19:16:40 on dist-test-slave-t7g5"
server_key: "42de65de52b6ee2d03d2769b95614370"
server_key_iv: "945bde7eb31b4eaf7834deb1e5cf97b1"
server_key_version: "encryptionkey@0"
I20260811 19:16:40.222074 1319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:16:40.249608 1319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:16:40.250605 1319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:16:40.251056 1319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:16:40.252221 1319 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:16:40.254422 1319 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:16:40.254527 1319 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:40.254640 1319 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:16:40.254716 1319 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:40.287017 1094 heartbeater.cc:503] Master 127.3.61.190:38067 was elected leader, sending a full tablet report...
I20260811 19:16:40.292219 1319 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.131:44789
I20260811 19:16:40.292256 1528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.131:44789 every 8 connection(s)
I20260811 19:16:40.293725 1319 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
I20260811 19:16:40.294700 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 1319
I20260811 19:16:40.294879 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/ts-2/wal/instance
I20260811 19:16:40.295514 3318 external_mini_cluster.cc:1428] Setting key 68f44ff4789cc40729f85cb1bf4b695a
I20260811 19:16:40.309634 1531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:40.296894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:39473 (local address 127.3.61.190:38067)
0811 19:16:40.297355 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:40.297361 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:40.299108 (+ 1747us) server_negotiation.cc:407] Connection header received
0811 19:16:40.301316 (+ 2208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:40.301320 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:40.301392 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:40.301499 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:16:40.302926 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:40.303864 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:40.306116 (+ 2252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:40.306683 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:40.306814 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:40.308718 (+ 1904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:16:40.308725 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:16:40.308729 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:16:40.308750 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:16:40.308802 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:16:40.308816 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:16:40.308890 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:40.309157 (+ 267us) server_negotiation.cc:299] Negotiation successful
0811 19:16:40.309270 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":231,"threads_started":1}
I20260811 19:16:40.311206 1532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:40.297743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:38067 (local address 127.3.61.131:39473)
0811 19:16:40.298924 (+ 1181us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:40.298985 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 19:16:40.300969 (+ 1984us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:16:40.301875 (+ 906us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:16:40.301884 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:16:40.301985 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:16:40.302628 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:40.302654 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:40.304165 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:40.304172 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:40.305906 (+ 1734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:40.305928 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:40.307075 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:40.307089 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:40.307377 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:40.308285 (+ 908us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 19:16:40.308288 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:16:40.308350 (+ 62us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 19:16:40.308463 (+ 113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:16:40.308953 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:16:40.308963 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:16:40.308971 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 19:16:40.309131 (+ 160us) client_negotiation.cc:240] Negotiation successful
0811 19:16:40.309494 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":320,"threads_started":1}
I20260811 19:16:40.312871 1529 heartbeater.cc:347] Connected to a master server at 127.3.61.190:38067
I20260811 19:16:40.313185 1529 heartbeater.cc:464] Registering TS with master...
I20260811 19:16:40.314126 1529 heartbeater.cc:511] Master 127.3.61.190:38067 requested a full tablet report, sending...
I20260811 19:16:40.315575 771 ts_manager.cc:194] Registered new tserver with Master: a0481b8231954a0ea2dde6d3dece7cce (127.3.61.131:44789)
I20260811 19:16:40.316385 771 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.61.131:39473
I20260811 19:16:40.325421 3318 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 19:16:40.336952 1531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:40.330355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60590 (local address 127.3.61.190:38067)
0811 19:16:40.330642 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:40.330649 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:40.330910 (+ 261us) server_negotiation.cc:407] Connection header received
0811 19:16:40.331101 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:40.331104 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:40.331174 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:40.331256 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:16:40.331886 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:40.332724 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:40.334215 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:40.334623 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:40.334739 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:40.336078 (+ 1339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:16:40.336085 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:16:40.336088 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:16:40.336116 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:16:40.336164 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:16:40.336178 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:16:40.336270 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:40.336424 (+ 154us) server_negotiation.cc:299] Negotiation successful
0811 19:16:40.336576 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260811 19:16:40.342423 771 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60590:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 19:16:40.351152 771 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:16:40.377471 1546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:40.366416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34106 (local address 127.3.61.130:41901)
0811 19:16:40.369767 (+ 3351us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:40.369776 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:40.369810 (+ 34us) server_negotiation.cc:407] Connection header received
0811 19:16:40.369920 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:40.369928 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:40.370184 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:40.370558 (+ 374us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:16:40.371090 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:40.372386 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:40.375089 (+ 2703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:40.376178 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:40.376319 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:40.376581 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:40.376879 (+ 298us) server_negotiation.cc:299] Negotiation successful
0811 19:16:40.377061 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3152,"thread_start_us":3089,"threads_started":1}
I20260811 19:16:40.377756 1545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:40.365901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.130:41901 (local address 127.0.0.1:34106)
0811 19:16:40.368068 (+ 2167us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:40.368085 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 19:16:40.368294 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:16:40.370417 (+ 2123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:16:40.370436 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:16:40.370464 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:16:40.370877 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:40.370889 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:40.372516 (+ 1627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:40.372521 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:40.374877 (+ 2356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:40.374896 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:40.376393 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:40.376399 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:40.376610 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:40.376664 (+ 54us) client_negotiation.cc:769] Sending connection context
0811 19:16:40.376811 (+ 147us) client_negotiation.cc:240] Negotiation successful
0811 19:16:40.377236 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2036,"mutex_wait_us":22,"thread_start_us":747,"threads_started":1}
I20260811 19:16:40.378561 1548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:40.368164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:38637 (local address 127.0.0.1:56998)
0811 19:16:40.369133 (+ 969us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:40.369149 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 19:16:40.369250 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:16:40.370385 (+ 1135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:16:40.370393 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:16:40.370421 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:16:40.371515 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:40.371532 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:40.373528 (+ 1996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:40.373533 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:40.375803 (+ 2270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:40.375818 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:40.377801 (+ 1983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:40.377807 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:40.377969 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:40.378001 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 19:16:40.378081 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 19:16:40.378234 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":849,"mutex_wait_us":3,"thread_start_us":710,"threads_started":1}
I20260811 19:16:40.378849 1550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:40.368541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56998 (local address 127.3.61.129:38637)
0811 19:16:40.369641 (+ 1100us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:40.369783 (+ 142us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:40.369822 (+ 39us) server_negotiation.cc:407] Connection header received
0811 19:16:40.369926 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:40.369932 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:40.370168 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:40.370360 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:16:40.371716 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:40.373343 (+ 1627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:40.376024 (+ 2681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:40.377380 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:40.377515 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:40.378343 (+ 828us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:40.378401 (+ 58us) server_negotiation.cc:299] Negotiation successful
0811 19:16:40.378532 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":752,"thread_start_us":611,"threads_started":1}
I20260811 19:16:40.380748 1544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:40.365802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.131:44789 (local address 127.0.0.1:36592)
0811 19:16:40.367168 (+ 1366us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:40.367229 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 19:16:40.367463 (+ 234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:16:40.371799 (+ 4336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:16:40.371803 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:16:40.371830 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:16:40.372169 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:40.372179 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:40.374175 (+ 1996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:40.374180 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:40.376238 (+ 2058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:40.376249 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:40.379990 (+ 3741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:40.379996 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:40.380170 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:40.380201 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 19:16:40.380297 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 19:16:40.380428 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1002,"mutex_wait_us":45,"thread_start_us":1950,"threads_started":1}
I20260811 19:16:40.380879 1547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:40.366716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36592 (local address 127.3.61.131:44789)
0811 19:16:40.371099 (+ 4383us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:40.371107 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:40.371143 (+ 36us) server_negotiation.cc:407] Connection header received
0811 19:16:40.371245 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:40.371253 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:40.371615 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:40.371799 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:16:40.372338 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:40.374017 (+ 1679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:40.376450 (+ 2433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:40.379791 (+ 3341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:40.379930 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:40.380182 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:40.380349 (+ 167us) server_negotiation.cc:299] Negotiation successful
0811 19:16:40.380507 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4202,"thread_start_us":4021,"threads_started":1}
I20260811 19:16:40.382020 1189 tablet_service.cc:1467] Processing CreateTablet for tablet a82a0af9afac4ab0b023d77178b8bfc1 (DEFAULT_TABLE table=test-table [id=e71e2ebd5ae542d29a5ae3840beac333]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:16:40.382238 988 tablet_service.cc:1467] Processing CreateTablet for tablet a82a0af9afac4ab0b023d77178b8bfc1 (DEFAULT_TABLE table=test-table [id=e71e2ebd5ae542d29a5ae3840beac333]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:16:40.383615 1189 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a82a0af9afac4ab0b023d77178b8bfc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:16:40.383610 988 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a82a0af9afac4ab0b023d77178b8bfc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:16:40.386128 1418 tablet_service.cc:1467] Processing CreateTablet for tablet a82a0af9afac4ab0b023d77178b8bfc1 (DEFAULT_TABLE table=test-table [id=e71e2ebd5ae542d29a5ae3840beac333]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:16:40.387449 1418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a82a0af9afac4ab0b023d77178b8bfc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:16:40.400686 1552 tablet_bootstrap.cc:492] T a82a0af9afac4ab0b023d77178b8bfc1 P a0481b8231954a0ea2dde6d3dece7cce: Bootstrap starting.
I20260811 19:16:40.403052 1559 tablet_bootstrap.cc:492] T a82a0af9afac4ab0b023d77178b8bfc1 P 2a96f8f035e349fdb3963bf6917a72f2: Bootstrap starting.
I20260811 19:16:40.405103 1552 tablet_bootstrap.cc:654] T a82a0af9afac4ab0b023d77178b8bfc1 P a0481b8231954a0ea2dde6d3dece7cce: Neither blocks nor log segments found. Creating new log.
I20260811 19:16:40.406699 1552 log.cc:824] T a82a0af9afac4ab0b023d77178b8bfc1 P a0481b8231954a0ea2dde6d3dece7cce: Log is configured to *not* fsync() on all Append() calls
I20260811 19:16:40.407248 1559 tablet_bootstrap.cc:654] T a82a0af9afac4ab0b023d77178b8bfc1 P 2a96f8f035e349fdb3963bf6917a72f2: Neither blocks nor log segments found. Creating new log.
I20260811 19:16:40.408645 1559 log.cc:824] T a82a0af9afac4ab0b023d77178b8bfc1 P 2a96f8f035e349fdb3963bf6917a72f2: Log is configured to *not* fsync() on all Append() calls
I20260811 19:16:40.410125 1556 tablet_bootstrap.cc:492] T a82a0af9afac4ab0b023d77178b8bfc1 P 3d14526964684cf6b179a72a679734c2: Bootstrap starting.
I20260811 19:16:40.412882 1559 tablet_bootstrap.cc:492] T a82a0af9afac4ab0b023d77178b8bfc1 P 2a96f8f035e349fdb3963bf6917a72f2: No bootstrap required, opened a new log
I20260811 19:16:40.413156 1559 ts_tablet_manager.cc:1397] T a82a0af9afac4ab0b023d77178b8bfc1 P 2a96f8f035e349fdb3963bf6917a72f2: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260811 19:16:40.413797 1552 tablet_bootstrap.cc:492] T a82a0af9afac4ab0b023d77178b8bfc1 P a0481b8231954a0ea2dde6d3dece7cce: No bootstrap required, opened a new log
I20260811 19:16:40.413905 1556 tablet_bootstrap.cc:654] T a82a0af9afac4ab0b023d77178b8bfc1 P 3d14526964684cf6b179a72a679734c2: Neither blocks nor log segments found. Creating new log.
I20260811 19:16:40.414085 1552 ts_tablet_manager.cc:1397] T a82a0af9afac4ab0b023d77178b8bfc1 P a0481b8231954a0ea2dde6d3dece7cce: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.006s
I20260811 19:16:40.415304 1556 log.cc:824] T a82a0af9afac4ab0b023d77178b8bfc1 P 3d14526964684cf6b179a72a679734c2: Log is configured to *not* fsync() on all Append() calls
I20260811 19:16:40.418654 1556 tablet_bootstrap.cc:492] T a82a0af9afac4ab0b023d77178b8bfc1 P 3d14526964684cf6b179a72a679734c2: No bootstrap required, opened a new log
I20260811 19:16:40.418949 1556 ts_tablet_manager.cc:1397] T a82a0af9afac4ab0b023d77178b8bfc1 P 3d14526964684cf6b179a72a679734c2: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260811 19:16:40.421952 1559 raft_consensus.cc:348] T a82a0af9afac4ab0b023d77178b8bfc1 P 2a96f8f035e349fdb3963bf6917a72f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0481b8231954a0ea2dde6d3dece7cce" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 44789 } } peers { permanent_uuid: "2a96f8f035e349fdb3963bf6917a72f2" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41901 } } peers { permanent_uuid: "3d14526964684cf6b179a72a679734c2" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 38637 } }
I20260811 19:16:40.422602 1559 raft_consensus.cc:374] T a82a0af9afac4ab0b023d77178b8bfc1 P 2a96f8f035e349fdb3963bf6917a72f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:16:40.422721 1559 raft_consensus.cc:729] T a82a0af9afac4ab0b023d77178b8bfc1 P 2a96f8f035e349fdb3963bf6917a72f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a96f8f035e349fdb3963bf6917a72f2, State: Initialized, Role: FOLLOWER
I20260811 19:16:40.423527 1559 consensus_queue.cc:260] T a82a0af9afac4ab0b023d77178b8bfc1 P 2a96f8f035e349fdb3963bf6917a72f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0481b8231954a0ea2dde6d3dece7cce" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 44789 } } peers { permanent_uuid: "2a96f8f035e349fdb3963bf6917a72f2" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41901 } } peers { permanent_uuid: "3d14526964684cf6b179a72a679734c2" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 38637 } }
I20260811 19:16:40.424947 1311 heartbeater.cc:503] Master 127.3.61.190:38067 was elected leader, sending a full tablet report...
I20260811 19:16:40.424437 1552 raft_consensus.cc:348] T a82a0af9afac4ab0b023d77178b8bfc1 P a0481b8231954a0ea2dde6d3dece7cce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0481b8231954a0ea2dde6d3dece7cce" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 44789 } } peers { permanent_uuid: "2a96f8f035e349fdb3963bf6917a72f2" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41901 } } peers { permanent_uuid: "3d14526964684cf6b179a72a679734c2" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 38637 } }
I20260811 19:16:40.425942 1559 ts_tablet_manager.cc:1428] T a82a0af9afac4ab0b023d77178b8bfc1 P 2a96f8f035e349fdb3963bf6917a72f2: Time spent starting tablet: real 0.013s user 0.010s sys 0.005s
I20260811 19:16:40.426196 1552 raft_consensus.cc:374] T a82a0af9afac4ab0b023d77178b8bfc1 P a0481b8231954a0ea2dde6d3dece7cce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:16:40.426424 1552 raft_consensus.cc:729] T a82a0af9afac4ab0b023d77178b8bfc1 P a0481b8231954a0ea2dde6d3dece7cce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0481b8231954a0ea2dde6d3dece7cce, State: Initialized, Role: FOLLOWER
I20260811 19:16:40.427572 1552 consensus_queue.cc:260] T a82a0af9afac4ab0b023d77178b8bfc1 P a0481b8231954a0ea2dde6d3dece7cce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0481b8231954a0ea2dde6d3dece7cce" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 44789 } } peers { permanent_uuid: "2a96f8f035e349fdb3963bf6917a72f2" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41901 } } peers { permanent_uuid: "3d14526964684cf6b179a72a679734c2" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 38637 } }
I20260811 19:16:40.429641 1556 raft_consensus.cc:348] T a82a0af9afac4ab0b023d77178b8bfc1 P 3d14526964684cf6b179a72a679734c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0481b8231954a0ea2dde6d3dece7cce" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 44789 } } peers { permanent_uuid: "2a96f8f035e349fdb3963bf6917a72f2" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41901 } } peers { permanent_uuid: "3d14526964684cf6b179a72a679734c2" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 38637 } }
I20260811 19:16:40.430459 1529 heartbeater.cc:503] Master 127.3.61.190:38067 was elected leader, sending a full tablet report...
I20260811 19:16:40.430476 1556 raft_consensus.cc:374] T a82a0af9afac4ab0b023d77178b8bfc1 P 3d14526964684cf6b179a72a679734c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:16:40.430603 1556 raft_consensus.cc:729] T a82a0af9afac4ab0b023d77178b8bfc1 P 3d14526964684cf6b179a72a679734c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d14526964684cf6b179a72a679734c2, State: Initialized, Role: FOLLOWER
I20260811 19:16:40.431330 1556 consensus_queue.cc:260] T a82a0af9afac4ab0b023d77178b8bfc1 P 3d14526964684cf6b179a72a679734c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0481b8231954a0ea2dde6d3dece7cce" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 44789 } } peers { permanent_uuid: "2a96f8f035e349fdb3963bf6917a72f2" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41901 } } peers { permanent_uuid: "3d14526964684cf6b179a72a679734c2" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 38637 } }
I20260811 19:16:40.431622 1552 ts_tablet_manager.cc:1428] T a82a0af9afac4ab0b023d77178b8bfc1 P a0481b8231954a0ea2dde6d3dece7cce: Time spent starting tablet: real 0.017s user 0.014s sys 0.002s
I20260811 19:16:40.434544 1556 ts_tablet_manager.cc:1428] T a82a0af9afac4ab0b023d77178b8bfc1 P 3d14526964684cf6b179a72a679734c2: Time spent starting tablet: real 0.015s user 0.012s sys 0.004s
I20260811 19:16:40.456945 1564 raft_consensus.cc:482] T a82a0af9afac4ab0b023d77178b8bfc1 P 2a96f8f035e349fdb3963bf6917a72f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:16:40.457356 1564 raft_consensus.cc:504] T a82a0af9afac4ab0b023d77178b8bfc1 P 2a96f8f035e349fdb3963bf6917a72f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0481b8231954a0ea2dde6d3dece7cce" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 44789 } } peers { permanent_uuid: "2a96f8f035e349fdb3963bf6917a72f2" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41901 } } peers { permanent_uuid: "3d14526964684cf6b179a72a679734c2" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 38637 } }
I20260811 19:16:40.459419 1564 leader_election.cc:288] T a82a0af9afac4ab0b023d77178b8bfc1 P 2a96f8f035e349fdb3963bf6917a72f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0481b8231954a0ea2dde6d3dece7cce (127.3.61.131:44789), 3d14526964684cf6b179a72a679734c2 (127.3.61.129:38637)
I20260811 19:16:40.467950 1547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:40.459891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:55799 (local address 127.3.61.131:44789)
0811 19:16:40.460071 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:40.460076 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:40.460709 (+ 633us) server_negotiation.cc:407] Connection header received
0811 19:16:40.460975 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:40.460983 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:40.461064 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:40.461126 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:16:40.462367 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:40.463144 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:40.465486 (+ 2342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:40.466849 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:40.467037 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:40.467112 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:40.467526 (+ 414us) server_negotiation.cc:299] Negotiation successful
0811 19:16:40.467694 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 19:16:40.468024 1567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:40.459780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.131:44789 (local address 127.3.61.130:55799)
0811 19:16:40.460569 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:40.460590 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 19:16:40.460754 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:16:40.461334 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:16:40.461337 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:16:40.461359 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:16:40.462047 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:40.462062 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:40.463374 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:40.463380 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:40.465215 (+ 1835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:40.465229 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:40.467095 (+ 1866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:40.467102 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:40.467288 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:40.467326 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 19:16:40.467450 (+ 124us) client_negotiation.cc:240] Negotiation successful
0811 19:16:40.467593 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"mutex_wait_us":1,"spinlock_wait_cycles":22912,"thread_start_us":316,"threads_started":1}
I20260811 19:16:40.469566 1550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:40.460826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:60289 (local address 127.3.61.129:38637)
0811 19:16:40.461044 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:40.461049 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:40.461415 (+ 366us) server_negotiation.cc:407] Connection header received
0811 19:16:40.461646 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:40.461689 (+ 43us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:40.461770 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:40.461848 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:16:40.462632 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:40.464296 (+ 1664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:40.467203 (+ 2907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:40.468410 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:40.468566 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:40.468632 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:40.469157 (+ 525us) server_negotiation.cc:299] Negotiation successful
0811 19:16:40.469311 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 19:16:40.469595 1569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:40.460676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:38637 (local address 127.3.61.130:60289)
0811 19:16:40.461294 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:40.461310 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 19:16:40.461511 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:16:40.461885 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:16:40.461889 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:16:40.461916 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:16:40.462439 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:40.462452 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:40.464627 (+ 2175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:40.464634 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:40.466968 (+ 2334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:40.466983 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:40.468769 (+ 1786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:40.468775 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:40.468969 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:40.469001 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 19:16:40.469094 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 19:16:40.469234 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"mutex_wait_us":59,"thread_start_us":280,"threads_started":1}
I20260811 19:16:40.469329 1458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a82a0af9afac4ab0b023d77178b8bfc1" candidate_uuid: "2a96f8f035e349fdb3963bf6917a72f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0481b8231954a0ea2dde6d3dece7cce" is_pre_election: true
I20260811 19:16:40.470054 1458 raft_consensus.cc:2454] T a82a0af9afac4ab0b023d77178b8bfc1 P a0481b8231954a0ea2dde6d3dece7cce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a96f8f035e349fdb3963bf6917a72f2 in term 0.
I20260811 19:16:40.470808 1126 leader_election.cc:302] T a82a0af9afac4ab0b023d77178b8bfc1 P 2a96f8f035e349fdb3963bf6917a72f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a96f8f035e349fdb3963bf6917a72f2, a0481b8231954a0ea2dde6d3dece7cce; no voters:
I20260811 19:16:40.470911 1022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a82a0af9afac4ab0b023d77178b8bfc1" candidate_uuid: "2a96f8f035e349fdb3963bf6917a72f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d14526964684cf6b179a72a679734c2" is_pre_election: true
I20260811 19:16:40.471342 1564 raft_consensus.cc:2781] T a82a0af9afac4ab0b023d77178b8bfc1 P 2a96f8f035e349fdb3963bf6917a72f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:16:40.471490 1564 raft_consensus.cc:482] T a82a0af9afac4ab0b023d77178b8bfc1 P 2a96f8f035e349fdb3963bf6917a72f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:16:40.471511 1022 raft_consensus.cc:2454] T a82a0af9afac4ab0b023d77178b8bfc1 P 3d14526964684cf6b179a72a679734c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a96f8f035e349fdb3963bf6917a72f2 in term 0.
I20260811 19:16:40.471598 1564 raft_consensus.cc:3037] T a82a0af9afac4ab0b023d77178b8bfc1 P 2a96f8f035e349fdb3963bf6917a72f2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:16:40.474498 1564 raft_consensus.cc:504] T a82a0af9afac4ab0b023d77178b8bfc1 P 2a96f8f035e349fdb3963bf6917a72f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0481b8231954a0ea2dde6d3dece7cce" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 44789 } } peers { permanent_uuid: "2a96f8f035e349fdb3963bf6917a72f2" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41901 } } peers { permanent_uuid: "3d14526964684cf6b179a72a679734c2" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 38637 } }
I20260811 19:16:40.475159 1564 leader_election.cc:288] T a82a0af9afac4ab0b023d77178b8bfc1 P 2a96f8f035e349fdb3963bf6917a72f2 [CANDIDATE]: Term 1 election: Requested vote from peers a0481b8231954a0ea2dde6d3dece7cce (127.3.61.131:44789), 3d14526964684cf6b179a72a679734c2 (127.3.61.129:38637)
I20260811 19:16:40.475746 1458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a82a0af9afac4ab0b023d77178b8bfc1" candidate_uuid: "2a96f8f035e349fdb3963bf6917a72f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0481b8231954a0ea2dde6d3dece7cce"
I20260811 19:16:40.475755 1022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a82a0af9afac4ab0b023d77178b8bfc1" candidate_uuid: "2a96f8f035e349fdb3963bf6917a72f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d14526964684cf6b179a72a679734c2"
I20260811 19:16:40.475975 1022 raft_consensus.cc:3037] T a82a0af9afac4ab0b023d77178b8bfc1 P 3d14526964684cf6b179a72a679734c2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:16:40.475975 1458 raft_consensus.cc:3037] T a82a0af9afac4ab0b023d77178b8bfc1 P a0481b8231954a0ea2dde6d3dece7cce [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:16:40.479423 1458 raft_consensus.cc:2454] T a82a0af9afac4ab0b023d77178b8bfc1 P a0481b8231954a0ea2dde6d3dece7cce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a96f8f035e349fdb3963bf6917a72f2 in term 1.
I20260811 19:16:40.479423 1022 raft_consensus.cc:2454] T a82a0af9afac4ab0b023d77178b8bfc1 P 3d14526964684cf6b179a72a679734c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a96f8f035e349fdb3963bf6917a72f2 in term 1.
I20260811 19:16:40.479990 1126 leader_election.cc:302] T a82a0af9afac4ab0b023d77178b8bfc1 P 2a96f8f035e349fdb3963bf6917a72f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a96f8f035e349fdb3963bf6917a72f2, a0481b8231954a0ea2dde6d3dece7cce; no voters:
I20260811 19:16:40.480566 1564 raft_consensus.cc:2781] T a82a0af9afac4ab0b023d77178b8bfc1 P 2a96f8f035e349fdb3963bf6917a72f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:16:40.481130 1564 raft_consensus.cc:686] T a82a0af9afac4ab0b023d77178b8bfc1 P 2a96f8f035e349fdb3963bf6917a72f2 [term 1 LEADER]: Becoming Leader. State: Replica: 2a96f8f035e349fdb3963bf6917a72f2, State: Running, Role: LEADER
I20260811 19:16:40.481748 1564 consensus_queue.cc:237] T a82a0af9afac4ab0b023d77178b8bfc1 P 2a96f8f035e349fdb3963bf6917a72f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0481b8231954a0ea2dde6d3dece7cce" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 44789 } } peers { permanent_uuid: "2a96f8f035e349fdb3963bf6917a72f2" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41901 } } peers { permanent_uuid: "3d14526964684cf6b179a72a679734c2" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 38637 } }
I20260811 19:16:40.487044 770 catalog_manager.cc:5495] T a82a0af9afac4ab0b023d77178b8bfc1 P 2a96f8f035e349fdb3963bf6917a72f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a96f8f035e349fdb3963bf6917a72f2 (127.3.61.130). New cstate: current_term: 1 leader_uuid: "2a96f8f035e349fdb3963bf6917a72f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0481b8231954a0ea2dde6d3dece7cce" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 44789 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a96f8f035e349fdb3963bf6917a72f2" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41901 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d14526964684cf6b179a72a679734c2" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 38637 } health_report { overall_health: UNKNOWN } } }
I20260811 19:16:40.506917 1546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:40.501151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34108 (local address 127.3.61.130:41901)
0811 19:16:40.501386 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:40.501391 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:40.501407 (+ 16us) server_negotiation.cc:407] Connection header received
0811 19:16:40.501499 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:40.501503 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:40.501594 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:40.501764 (+ 170us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:16:40.502126 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:40.503021 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:40.504583 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:40.505033 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:40.505149 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:40.505716 (+ 567us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:16:40.506282 (+ 566us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:16:40.506434 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:40.506534 (+ 100us) server_negotiation.cc:299] Negotiation successful
0811 19:16:40.506624 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260811 19:16:40.512929 1095 tablet.cc:2369] T a82a0af9afac4ab0b023d77178b8bfc1 P 3d14526964684cf6b179a72a679734c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:16:40.516595 1022 raft_consensus.cc:1261] T a82a0af9afac4ab0b023d77178b8bfc1 P 3d14526964684cf6b179a72a679734c2 [term 1 FOLLOWER]: Refusing update from remote peer 2a96f8f035e349fdb3963bf6917a72f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:16:40.516650 1458 raft_consensus.cc:1261] T a82a0af9afac4ab0b023d77178b8bfc1 P a0481b8231954a0ea2dde6d3dece7cce [term 1 FOLLOWER]: Refusing update from remote peer 2a96f8f035e349fdb3963bf6917a72f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:16:40.517746 1564 consensus_queue.cc:1035] T a82a0af9afac4ab0b023d77178b8bfc1 P 2a96f8f035e349fdb3963bf6917a72f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d14526964684cf6b179a72a679734c2" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 38637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:16:40.518203 1570 consensus_queue.cc:1035] T a82a0af9afac4ab0b023d77178b8bfc1 P 2a96f8f035e349fdb3963bf6917a72f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0481b8231954a0ea2dde6d3dece7cce" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 44789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 19:16:40.525893 1312 tablet.cc:2369] T a82a0af9afac4ab0b023d77178b8bfc1 P 2a96f8f035e349fdb3963bf6917a72f2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:16:40.531421 1576 mvcc.cc:205] Tried to move back new op lower bound from 7317404878905475072 to 7317404878777909248. Current Snapshot: MvccSnapshot[applied={T|T < 7317404878905475072}]
I20260811 19:16:40.533706 1579 mvcc.cc:205] Tried to move back new op lower bound from 7317404878905475072 to 7317404878777909248. Current Snapshot: MvccSnapshot[applied={T|T < 7317404878905475072}]
I20260811 19:16:40.546811 771 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60590:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:16:40.547132 771 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60590:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:16:40.551075 771 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fa27dd66f1c741b1ae5199df9ca13e6a: Sending DeleteTablet for 3 replicas of tablet a82a0af9afac4ab0b023d77178b8bfc1
I20260811 19:16:40.552425 1418 tablet_service.cc:1514] Processing DeleteTablet for tablet a82a0af9afac4ab0b023d77178b8bfc1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:16:40 UTC) from {username='slave'} at 127.0.0.1:36592
I20260811 19:16:40.552444 1189 tablet_service.cc:1514] Processing DeleteTablet for tablet a82a0af9afac4ab0b023d77178b8bfc1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:16:40 UTC) from {username='slave'} at 127.0.0.1:34106
I20260811 19:16:40.554143 988 tablet_service.cc:1514] Processing DeleteTablet for tablet a82a0af9afac4ab0b023d77178b8bfc1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:16:40 UTC) from {username='slave'} at 127.0.0.1:56998
I20260811 19:16:40.554921 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 893
I20260811 19:16:40.566330 1596 tablet_replica.cc:330] T a82a0af9afac4ab0b023d77178b8bfc1 P 3d14526964684cf6b179a72a679734c2: stopping tablet replica
I20260811 19:16:40.567660 1596 raft_consensus.cc:2229] T a82a0af9afac4ab0b023d77178b8bfc1 P 3d14526964684cf6b179a72a679734c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:16:40.568177 1593 tablet_replica.cc:330] T a82a0af9afac4ab0b023d77178b8bfc1 P a0481b8231954a0ea2dde6d3dece7cce: stopping tablet replica
I20260811 19:16:40.568594 1596 raft_consensus.cc:2258] T a82a0af9afac4ab0b023d77178b8bfc1 P 3d14526964684cf6b179a72a679734c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:16:40.568615 1593 raft_consensus.cc:2229] T a82a0af9afac4ab0b023d77178b8bfc1 P a0481b8231954a0ea2dde6d3dece7cce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:16:40.569064 1593 raft_consensus.cc:2258] T a82a0af9afac4ab0b023d77178b8bfc1 P a0481b8231954a0ea2dde6d3dece7cce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:16:40.569849 1594 tablet_replica.cc:330] T a82a0af9afac4ab0b023d77178b8bfc1 P 2a96f8f035e349fdb3963bf6917a72f2: stopping tablet replica
I20260811 19:16:40.570256 1594 raft_consensus.cc:2229] T a82a0af9afac4ab0b023d77178b8bfc1 P 2a96f8f035e349fdb3963bf6917a72f2 [term 1 LEADER]: Raft consensus shutting down.
I20260811 19:16:40.570963 1594 raft_consensus.cc:2258] T a82a0af9afac4ab0b023d77178b8bfc1 P 2a96f8f035e349fdb3963bf6917a72f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:16:40.571797 1596 ts_tablet_manager.cc:1905] T a82a0af9afac4ab0b023d77178b8bfc1 P 3d14526964684cf6b179a72a679734c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:16:40.573781 1593 ts_tablet_manager.cc:1905] T a82a0af9afac4ab0b023d77178b8bfc1 P a0481b8231954a0ea2dde6d3dece7cce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:16:40.574144 1550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:40.556325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57004 (local address 127.3.61.129:38637)
0811 19:16:40.556516 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:40.556523 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:40.565299 (+ 8776us) server_negotiation.cc:407] Connection header received
0811 19:16:40.565521 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:40.565529 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:40.565703 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:40.565836 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:16:40.566703 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:40.567951 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:40.569900 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:40.570535 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:40.570634 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:40.572184 (+ 1550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:16:40.572196 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:16:40.572215 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:16:40.573011 (+ 796us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:16:40.573079 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:16:40.573191 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:16:40.573302 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:40.573600 (+ 298us) server_negotiation.cc:299] Negotiation successful
0811 19:16:40.573759 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 19:16:40.575409 1085 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:16:40.582048 1593 ts_tablet_manager.cc:1918] T a82a0af9afac4ab0b023d77178b8bfc1 P a0481b8231954a0ea2dde6d3dece7cce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:16:40.582267 1593 log.cc:1196] T a82a0af9afac4ab0b023d77178b8bfc1 P a0481b8231954a0ea2dde6d3dece7cce: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/ts-2/wal/wals/a82a0af9afac4ab0b023d77178b8bfc1
I20260811 19:16:40.582870 1593 ts_tablet_manager.cc:1939] T a82a0af9afac4ab0b023d77178b8bfc1 P a0481b8231954a0ea2dde6d3dece7cce: Deleting consensus metadata
I20260811 19:16:40.584731 1594 ts_tablet_manager.cc:1905] T a82a0af9afac4ab0b023d77178b8bfc1 P 2a96f8f035e349fdb3963bf6917a72f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:16:40.585156 751 catalog_manager.cc:4841] TS a0481b8231954a0ea2dde6d3dece7cce (127.3.61.131:44789): tablet a82a0af9afac4ab0b023d77178b8bfc1 (table test-table [id=e71e2ebd5ae542d29a5ae3840beac333]) successfully deleted
I20260811 19:16:40.591274 1594 ts_tablet_manager.cc:1918] T a82a0af9afac4ab0b023d77178b8bfc1 P 2a96f8f035e349fdb3963bf6917a72f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:16:40.591482 1594 log.cc:1196] T a82a0af9afac4ab0b023d77178b8bfc1 P 2a96f8f035e349fdb3963bf6917a72f2: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786475454804547-3318-0/minicluster-data/ts-1/wal/wals/a82a0af9afac4ab0b023d77178b8bfc1
I20260811 19:16:40.591959 1594 ts_tablet_manager.cc:1939] T a82a0af9afac4ab0b023d77178b8bfc1 P 2a96f8f035e349fdb3963bf6917a72f2: Deleting consensus metadata
I20260811 19:16:40.593689 748 catalog_manager.cc:4841] TS 2a96f8f035e349fdb3963bf6917a72f2 (127.3.61.130:41901): tablet a82a0af9afac4ab0b023d77178b8bfc1 (table test-table [id=e71e2ebd5ae542d29a5ae3840beac333]) successfully deleted
I20260811 19:16:40.702370 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 893
W20260811 19:16:40.729683 751 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.61.129:38637 (error 108)
I20260811 19:16:40.729830 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 1102
I20260811 19:16:40.731931 1544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:40.731395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:38637 (local address 127.0.0.1:57012)
0811 19:16:40.731636 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:40.731758 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.129:38637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134}
W20260811 19:16:40.732213 751 catalog_manager.cc:4584] TS 3d14526964684cf6b179a72a679734c2 (127.3.61.129:38637): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a82a0af9afac4ab0b023d77178b8bfc1: Network error: Client connection negotiation failed: client connection to 127.3.61.129:38637: connect: Connection refused (error 111)
I20260811 19:16:40.737504 1546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:40.730942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34112 (local address 127.3.61.130:41901)
0811 19:16:40.731128 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:40.731133 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:40.731149 (+ 16us) server_negotiation.cc:407] Connection header received
0811 19:16:40.731209 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:40.731212 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:40.731314 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:40.731410 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:16:40.731883 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:40.732855 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:40.734289 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:40.734727 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:40.734838 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:40.735979 (+ 1141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:16:40.735989 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:16:40.736009 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:16:40.736669 (+ 660us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:16:40.736721 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:16:40.736748 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:16:40.736851 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:40.737008 (+ 157us) server_negotiation.cc:299] Negotiation successful
0811 19:16:40.737122 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 19:16:40.738821 1304 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:16:40.786170 1544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:40.785642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:38637 (local address 127.0.0.1:57018)
0811 19:16:40.785899 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:40.786000 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.129:38637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260811 19:16:40.787300 1544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:40.786943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:38637 (local address 127.0.0.1:57022)
0811 19:16:40.787090 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:40.787162 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.129:38637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260811 19:16:40.848400 1544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:40.847739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:38637 (local address 127.0.0.1:57026)
0811 19:16:40.848047 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:40.848164 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.129:38637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126}
I20260811 19:16:40.850209 1544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:40.849484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:38637 (local address 127.0.0.1:57040)
0811 19:16:40.849827 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:40.849985 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.129:38637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260811 19:16:40.864387 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 1102
I20260811 19:16:40.897951 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 1319
I20260811 19:16:40.905596 1547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:40.898860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36602 (local address 127.3.61.131:44789)
0811 19:16:40.899097 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:40.899104 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:40.899187 (+ 83us) server_negotiation.cc:407] Connection header received
0811 19:16:40.899391 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:40.899395 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:40.899486 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:40.899587 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:16:40.899977 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:40.900723 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:40.902149 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:40.902559 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:40.902675 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:40.904118 (+ 1443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:16:40.904129 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:16:40.904147 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:16:40.904776 (+ 629us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:16:40.904832 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:16:40.904853 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:16:40.904974 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:40.905116 (+ 142us) server_negotiation.cc:299] Negotiation successful
0811 19:16:40.905248 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 19:16:40.906554 1520 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:16:40.931139 1544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:40.930550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:38637 (local address 127.0.0.1:57056)
0811 19:16:40.930819 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:40.930943 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.129:38637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260811 19:16:40.933102 1544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:40.932340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:38637 (local address 127.0.0.1:57062)
0811 19:16:40.932809 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:40.932927 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.129:38637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226}
I20260811 19:16:41.003629 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 1319
I20260811 19:16:41.034264 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 725
I20260811 19:16:41.035713 835 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:16:41.154354 1544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:41.153369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:38637 (local address 127.0.0.1:57068)
0811 19:16:41.153577 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:41.153753 (+ 176us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.129:38637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260811 19:16:41.154855 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 725
2026-08-11T19:16:41Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (3213 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:16:41 dist-test-slave-t7g5 krb5kdc[1613](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:16:41 dist-test-slave-t7g5 krb5kdc[1613](info): set up 2 sockets
Aug 11 19:16:41 dist-test-slave-t7g5 krb5kdc[1613](info): commencing operation
krb5kdc: starting...
W20260811 19:16:42.495886 3318 mini_kdc.cc:121] Time spent starting KDC: real 1.280s user 0.004s sys 0.057s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Aug 11 19:16:42 dist-test-slave-t7g5 krb5kdc[1613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475802, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:16:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:16:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.61.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab.
Entry for principal kudu/127.3.61.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab.
I20260811 19:16:42.877511 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:44027
--webserver_interface=127.3.61.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:35339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.61.190:44027
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:16:43.263937 1634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:16:43.264303 1634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:16:43.264446 1634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:16:43.273142 1634 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:16:43.273331 1634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:16:43.273422 1634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:16:43.273504 1634 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:16:43.273563 1634 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:16:43.286608 1634 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:35339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.61.190:44027
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:44027
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.61.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:16:43.288249 1634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:16:43.289724 1634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:16:43.303256 1640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:16:43.303397 1639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:16:43.303587 1643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:16:43.304997 1634 server_base.cc:1034] running on GCE node
I20260811 19:16:43.306166 1634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:16:43.307801 1634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:16:43.309038 1634 hybrid_clock.cc:648] HybridClock initialized: now 1786475803308973 us; error 67 us; skew 500 ppm
I20260811 19:16:43.309474 1634 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:16:43 dist-test-slave-t7g5 krb5kdc[1613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475803, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:16:43.313324 1634 init.cc:377] Logged in from keytab as kudu/127.3.61.190@KRBTEST.COM (short username kudu)
I20260811 19:16:43.315265 1634 webserver.cc:460] Webserver started at http://127.3.61.190:44049/ using document root <none> and password file <none>
I20260811 19:16:43.316076 1634 fs_manager.cc:359] Metadata directory not provided
I20260811 19:16:43.316217 1634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:16:43.316569 1634 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:16:43.320683 1634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/master-0/data/instance:
uuid: "9a5aecaa6f75402586c7137edebc6757"
format_stamp: "Formatted at 2026-08-11 19:16:43 on dist-test-slave-t7g5"
server_key: "988bb88fe2830d9208551ee73622343e"
server_key_iv: "605793db264e291f09993c13a32fb606"
server_key_version: "encryptionkey@0"
I20260811 19:16:43.321574 1634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/master-0/wal/instance:
uuid: "9a5aecaa6f75402586c7137edebc6757"
format_stamp: "Formatted at 2026-08-11 19:16:43 on dist-test-slave-t7g5"
server_key: "988bb88fe2830d9208551ee73622343e"
server_key_iv: "605793db264e291f09993c13a32fb606"
server_key_version: "encryptionkey@0"
I20260811 19:16:43.327422 1634 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 19:16:43.332710 1651 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:43.334689 1634 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260811 19:16:43.334909 1634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/master-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/master-0/wal
uuid: "9a5aecaa6f75402586c7137edebc6757"
format_stamp: "Formatted at 2026-08-11 19:16:43 on dist-test-slave-t7g5"
server_key: "988bb88fe2830d9208551ee73622343e"
server_key_iv: "605793db264e291f09993c13a32fb606"
server_key_version: "encryptionkey@0"
I20260811 19:16:43.335142 1634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:16:43.366773 1634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:16:43.373508 1634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:16:43.373963 1634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:16:43.396294 1634 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.190:44027
I20260811 19:16:43.396373 1732 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.190:44027 every 8 connection(s)
I20260811 19:16:43.398438 1634 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
I20260811 19:16:43.400708 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 1634
I20260811 19:16:43.400909 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/master-0/wal/instance
I20260811 19:16:43.401387 3318 external_mini_cluster.cc:1428] Setting key b2a192a5c8a927b8227f34cd1c081e14
I20260811 19:16:43.404872 1733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 19:16:43 dist-test-slave-t7g5 krb5kdc[1613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475802, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:16:43.419831 1733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a5aecaa6f75402586c7137edebc6757: Bootstrap starting.
I20260811 19:16:43.425844 1733 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9a5aecaa6f75402586c7137edebc6757: Neither blocks nor log segments found. Creating new log.
I20260811 19:16:43.427601 1733 log.cc:824] T 00000000000000000000000000000000 P 9a5aecaa6f75402586c7137edebc6757: Log is configured to *not* fsync() on all Append() calls
I20260811 19:16:43.431619 1736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:43.403575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35666 (local address 127.3.61.190:44027)
0811 19:16:43.404448 (+ 873us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:43.404462 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:43.404512 (+ 50us) server_negotiation.cc:407] Connection header received
0811 19:16:43.406371 (+ 1859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:43.406395 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:43.406809 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:43.407167 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:16:43.409132 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:43.411019 (+ 1887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:43.413182 (+ 2163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:43.414299 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:43.414520 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:43.419959 (+ 5439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:16:43.419981 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:16:43.420017 (+ 36us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:16:43.420119 (+ 102us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:16:43.425140 (+ 5021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:16:43.426306 (+ 1166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:16:43.426326 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:16:43.426339 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:16:43.426682 (+ 343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:16:43.427233 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:16:43.427238 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:16:43.427242 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:16:43.428180 (+ 938us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:16:43.428728 (+ 548us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:43.429520 (+ 792us) server_negotiation.cc:299] Negotiation successful
0811 19:16:43.430136 (+ 616us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":651,"thread_start_us":369,"threads_started":1}
I20260811 19:16:43.432459 1733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a5aecaa6f75402586c7137edebc6757: No bootstrap required, opened a new log
I20260811 19:16:43.441207 1733 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a5aecaa6f75402586c7137edebc6757 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a5aecaa6f75402586c7137edebc6757" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 44027 } }
I20260811 19:16:43.441638 1733 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9a5aecaa6f75402586c7137edebc6757 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:16:43.441798 1733 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a5aecaa6f75402586c7137edebc6757 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a5aecaa6f75402586c7137edebc6757, State: Initialized, Role: FOLLOWER
I20260811 19:16:43.442408 1733 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a5aecaa6f75402586c7137edebc6757 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a5aecaa6f75402586c7137edebc6757" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 44027 } }
I20260811 19:16:43.442636 1733 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9a5aecaa6f75402586c7137edebc6757 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:16:43.442744 1733 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a5aecaa6f75402586c7137edebc6757 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:16:43.442893 1733 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a5aecaa6f75402586c7137edebc6757 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:16:43.445161 1733 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a5aecaa6f75402586c7137edebc6757 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a5aecaa6f75402586c7137edebc6757" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 44027 } }
I20260811 19:16:43.445722 1733 leader_election.cc:302] T 00000000000000000000000000000000 P 9a5aecaa6f75402586c7137edebc6757 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9a5aecaa6f75402586c7137edebc6757; no voters:
I20260811 19:16:43.446367 1733 leader_election.cc:288] T 00000000000000000000000000000000 P 9a5aecaa6f75402586c7137edebc6757 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:16:43.446559 1740 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a5aecaa6f75402586c7137edebc6757 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:16:43.447660 1740 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a5aecaa6f75402586c7137edebc6757 [term 1 LEADER]: Becoming Leader. State: Replica: 9a5aecaa6f75402586c7137edebc6757, State: Running, Role: LEADER
I20260811 19:16:43.448272 1740 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a5aecaa6f75402586c7137edebc6757 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a5aecaa6f75402586c7137edebc6757" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 44027 } }
I20260811 19:16:43.448825 1733 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a5aecaa6f75402586c7137edebc6757 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:16:43.451164 1742 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a5aecaa6f75402586c7137edebc6757 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9a5aecaa6f75402586c7137edebc6757" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a5aecaa6f75402586c7137edebc6757" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 44027 } } }
I20260811 19:16:43.451076 1743 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a5aecaa6f75402586c7137edebc6757 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a5aecaa6f75402586c7137edebc6757. Latest consensus state: current_term: 1 leader_uuid: "9a5aecaa6f75402586c7137edebc6757" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a5aecaa6f75402586c7137edebc6757" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 44027 } } }
I20260811 19:16:43.451611 1743 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a5aecaa6f75402586c7137edebc6757 [sys.catalog]: This master's current role is: LEADER
I20260811 19:16:43.451611 1742 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a5aecaa6f75402586c7137edebc6757 [sys.catalog]: This master's current role is: LEADER
I20260811 19:16:43.452579 1755 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:16:43.458757 1755 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:16:43.469628 1755 catalog_manager.cc:1347] Generated new cluster ID: 4944534c5bca4508a9b737a5172fdd32
I20260811 19:16:43.469810 1755 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:16:43.489735 1755 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:16:43.491153 1755 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:16:43.497743 1755 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9a5aecaa6f75402586c7137edebc6757: Generated new TSK 0
I20260811 19:16:43.498855 1755 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.61.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab.
Entry for principal kudu/127.3.61.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab.
I20260811 19:16:43.585315 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.129:0
--local_ip_for_outbound_sockets=127.3.61.129
--webserver_interface=127.3.61.129
--webserver_port=0
--tserver_master_addrs=127.3.61.190:44027
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab
--principal=kudu/127.3.61.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:35339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:16:43.991596 1778 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:16:43.991930 1778 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:16:43.992061 1778 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:16:44.000216 1778 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:16:44.000371 1778 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:16:44.000895 1778 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.129
I20260811 19:16:44.013895 1778 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:35339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab
--principal=kudu/127.3.61.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.61.129
--webserver_port=0
--tserver_master_addrs=127.3.61.190:44027
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.129
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:16:44.015314 1778 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:16:44.016721 1778 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:16:44.032625 1797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:16:44.032675 1793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:16:44.033286 1792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:16:44.033877 1778 server_base.cc:1034] running on GCE node
I20260811 19:16:44.034679 1778 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:16:44.035830 1778 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:16:44.037078 1778 hybrid_clock.cc:648] HybridClock initialized: now 1786475804036977 us; error 93 us; skew 500 ppm
I20260811 19:16:44.037550 1778 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:16:44 dist-test-slave-t7g5 krb5kdc[1613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475804, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:16:44.042114 1778 init.cc:377] Logged in from keytab as kudu/127.3.61.129@KRBTEST.COM (short username kudu)
I20260811 19:16:44.044159 1778 webserver.cc:460] Webserver started at http://127.3.61.129:32981/ using document root <none> and password file <none>
I20260811 19:16:44.045343 1778 fs_manager.cc:359] Metadata directory not provided
I20260811 19:16:44.045495 1778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:16:44.046049 1778 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:16:44.050632 1778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/ts-0/data/instance:
uuid: "090c784003f544b5871e8615fea2ff51"
format_stamp: "Formatted at 2026-08-11 19:16:44 on dist-test-slave-t7g5"
server_key: "7871410ae6e8c13463f5b3e6153ce07b"
server_key_iv: "12831dc3cb83ba2f7d44b5ffee7c0d77"
server_key_version: "encryptionkey@0"
I20260811 19:16:44.051864 1778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/ts-0/wal/instance:
uuid: "090c784003f544b5871e8615fea2ff51"
format_stamp: "Formatted at 2026-08-11 19:16:44 on dist-test-slave-t7g5"
server_key: "7871410ae6e8c13463f5b3e6153ce07b"
server_key_iv: "12831dc3cb83ba2f7d44b5ffee7c0d77"
server_key_version: "encryptionkey@0"
I20260811 19:16:44.058277 1778 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260811 19:16:44.063058 1812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:44.065519 1778 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.000s
I20260811 19:16:44.065774 1778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/ts-0/wal
uuid: "090c784003f544b5871e8615fea2ff51"
format_stamp: "Formatted at 2026-08-11 19:16:44 on dist-test-slave-t7g5"
server_key: "7871410ae6e8c13463f5b3e6153ce07b"
server_key_iv: "12831dc3cb83ba2f7d44b5ffee7c0d77"
server_key_version: "encryptionkey@0"
I20260811 19:16:44.066035 1778 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:16:44.094135 1778 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:16:44.098953 1778 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:16:44.099242 1778 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:16:44.100343 1778 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:16:44.102077 1778 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:16:44.102164 1778 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:44.102265 1778 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:16:44.102336 1778 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:44.141896 1778 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.129:32911
I20260811 19:16:44.141968 1997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.129:32911 every 8 connection(s)
I20260811 19:16:44.143343 1778 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
I20260811 19:16:44.145267 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 1778
I20260811 19:16:44.145448 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/ts-0/wal/instance
I20260811 19:16:44.145953 3318 external_mini_cluster.cc:1428] Setting key 525b6b20ccc2eb1e49df99cc3f16ca51
Aug 11 19:16:44 dist-test-slave-t7g5 krb5kdc[1613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475804, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.129@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:16:44.168817 2000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:44.146949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:42203 (local address 127.3.61.190:44027)
0811 19:16:44.147484 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:44.147490 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:44.148742 (+ 1252us) server_negotiation.cc:407] Connection header received
0811 19:16:44.150922 (+ 2180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:44.150929 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:44.151017 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:44.151115 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:16:44.152830 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:44.153614 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:44.155356 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:44.155793 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:44.155911 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:44.159999 (+ 4088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:16:44.160006 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:16:44.160009 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:16:44.160047 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:16:44.163369 (+ 3322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:16:44.165146 (+ 1777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:16:44.165155 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:16:44.165159 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:16:44.165435 (+ 276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:16:44.166127 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:16:44.166133 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:16:44.166136 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:16:44.166649 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:16:44.166871 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:44.168004 (+ 1133us) server_negotiation.cc:299] Negotiation successful
0811 19:16:44.168364 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":277,"threads_started":1}
I20260811 19:16:44.170300 2004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:44.147597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:44027 (local address 127.3.61.129:42203)
0811 19:16:44.148570 (+ 973us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:44.148645 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0811 19:16:44.150629 (+ 1984us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:16:44.151388 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:16:44.151398 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:16:44.151966 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:16:44.152620 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:44.152641 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:44.153813 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:44.153817 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:44.155185 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:44.155206 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:44.156040 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:44.156049 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:44.156292 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:44.157049 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:16:44.157075 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:16:44.159686 (+ 2611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:16:44.163799 (+ 4113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:16:44.163816 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:16:44.163845 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:16:44.164861 (+ 1016us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:16:44.165692 (+ 831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:16:44.165698 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:16:44.165702 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:16:44.165992 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:16:44.166890 (+ 898us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:16:44.166901 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:16:44.167461 (+ 560us) client_negotiation.cc:769] Sending connection context
0811 19:16:44.167813 (+ 352us) client_negotiation.cc:240] Negotiation successful
0811 19:16:44.168450 (+ 637us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":613,"thread_start_us":371,"threads_started":1}
I20260811 19:16:44.172765 1998 heartbeater.cc:347] Connected to a master server at 127.3.61.190:44027
I20260811 19:16:44.173228 1998 heartbeater.cc:464] Registering TS with master...
I20260811 19:16:44.174340 1998 heartbeater.cc:511] Master 127.3.61.190:44027 requested a full tablet report, sending...
I20260811 19:16:44.178117 1681 ts_manager.cc:194] Registered new tserver with Master: 090c784003f544b5871e8615fea2ff51 (127.3.61.129:32911)
I20260811 19:16:44.179713 1681 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.129@KRBTEST.COM'} at 127.3.61.129:42203
WARNING: no policy specified for kudu/127.3.61.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab.
Entry for principal kudu/127.3.61.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab.
I20260811 19:16:44.237483 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.130:0
--local_ip_for_outbound_sockets=127.3.61.130
--webserver_interface=127.3.61.130
--webserver_port=0
--tserver_master_addrs=127.3.61.190:44027
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab
--principal=kudu/127.3.61.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:35339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:16:44.619546 2008 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:16:44.619992 2008 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:16:44.620177 2008 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:16:44.629055 2008 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:16:44.629194 2008 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:16:44.629319 2008 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.130
I20260811 19:16:44.642221 2008 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:35339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab
--principal=kudu/127.3.61.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.61.130
--webserver_port=0
--tserver_master_addrs=127.3.61.190:44027
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.130
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:16:44.643599 2008 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:16:44.644906 2008 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:16:44.658502 2015 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:16:44.658429 2017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:16:44.658473 2019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:16:44.659646 2008 server_base.cc:1034] running on GCE node
I20260811 19:16:44.660281 2008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:16:44.661296 2008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:16:44.662751 2008 hybrid_clock.cc:648] HybridClock initialized: now 1786475804662655 us; error 119 us; skew 500 ppm
I20260811 19:16:44.663106 2008 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:16:44 dist-test-slave-t7g5 krb5kdc[1613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475804, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:16:44.667032 2008 init.cc:377] Logged in from keytab as kudu/127.3.61.130@KRBTEST.COM (short username kudu)
I20260811 19:16:44.668812 2008 webserver.cc:460] Webserver started at http://127.3.61.130:35403/ using document root <none> and password file <none>
I20260811 19:16:44.669569 2008 fs_manager.cc:359] Metadata directory not provided
I20260811 19:16:44.669719 2008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:16:44.670044 2008 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:16:44.673148 2008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/ts-1/data/instance:
uuid: "1562ddf0a7b94143a43c7bc580388375"
format_stamp: "Formatted at 2026-08-11 19:16:44 on dist-test-slave-t7g5"
server_key: "b44c8d854190f91df0d074cb07660088"
server_key_iv: "c1602d331f4ac4d068b064602b7c98f9"
server_key_version: "encryptionkey@0"
I20260811 19:16:44.674008 2008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/ts-1/wal/instance:
uuid: "1562ddf0a7b94143a43c7bc580388375"
format_stamp: "Formatted at 2026-08-11 19:16:44 on dist-test-slave-t7g5"
server_key: "b44c8d854190f91df0d074cb07660088"
server_key_iv: "c1602d331f4ac4d068b064602b7c98f9"
server_key_version: "encryptionkey@0"
I20260811 19:16:44.679350 2008 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 19:16:44.683189 2029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:44.684757 2008 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260811 19:16:44.684928 2008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/ts-1/wal
uuid: "1562ddf0a7b94143a43c7bc580388375"
format_stamp: "Formatted at 2026-08-11 19:16:44 on dist-test-slave-t7g5"
server_key: "b44c8d854190f91df0d074cb07660088"
server_key_iv: "c1602d331f4ac4d068b064602b7c98f9"
server_key_version: "encryptionkey@0"
I20260811 19:16:44.685123 2008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:16:44.726399 2008 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:16:44.732138 2008 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:16:44.732483 2008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:16:44.733628 2008 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:16:44.735302 2008 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:16:44.735385 2008 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:44.735527 2008 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:16:44.735589 2008 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:44.773382 2008 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.130:40981
I20260811 19:16:44.773437 2212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.130:40981 every 8 connection(s)
I20260811 19:16:44.774784 2008 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
I20260811 19:16:44.778801 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 2008
I20260811 19:16:44.779011 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/ts-1/wal/instance
I20260811 19:16:44.779585 3318 external_mini_cluster.cc:1428] Setting key 9e66a7af6bbad337dafa5ee12d4c2aa2
Aug 11 19:16:44 dist-test-slave-t7g5 krb5kdc[1613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475804, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.130@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:16:44.802934 2216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:44.777507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:57953 (local address 127.3.61.190:44027)
0811 19:16:44.778052 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:44.778056 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:44.779899 (+ 1843us) server_negotiation.cc:407] Connection header received
0811 19:16:44.782934 (+ 3035us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:44.782938 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:44.783024 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:44.783137 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:16:44.786244 (+ 3107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:44.787070 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:44.789884 (+ 2814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:44.790416 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:44.790533 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:44.795498 (+ 4965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:16:44.795505 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:16:44.795510 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:16:44.795553 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:16:44.798592 (+ 3039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:16:44.799612 (+ 1020us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:16:44.799618 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:16:44.799620 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:16:44.799782 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:16:44.800271 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:16:44.800276 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:16:44.800279 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:16:44.800670 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:16:44.801029 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:44.802298 (+ 1269us) server_negotiation.cc:299] Negotiation successful
0811 19:16:44.802547 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":266,"threads_started":1}
I20260811 19:16:44.804323 2217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:44.778065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:44027 (local address 127.3.61.130:57953)
0811 19:16:44.779697 (+ 1632us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:44.779759 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 19:16:44.782412 (+ 2653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:16:44.783564 (+ 1152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:16:44.783579 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:16:44.784819 (+ 1240us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:16:44.785968 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:44.786006 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:44.787335 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:44.787342 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:44.789594 (+ 2252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:44.789619 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:44.790678 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:44.790686 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:44.790987 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:44.791935 (+ 948us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:16:44.791967 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:16:44.795091 (+ 3124us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:16:44.798781 (+ 3690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:16:44.798790 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:16:44.798809 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:16:44.799446 (+ 637us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:16:44.799904 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:16:44.799908 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:16:44.799910 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:16:44.800106 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:16:44.801041 (+ 935us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:16:44.801053 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:16:44.801738 (+ 685us) client_negotiation.cc:769] Sending connection context
0811 19:16:44.802140 (+ 402us) client_negotiation.cc:240] Negotiation successful
0811 19:16:44.802887 (+ 747us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1183,"thread_start_us":894,"threads_started":1}
I20260811 19:16:44.806156 2214 heartbeater.cc:347] Connected to a master server at 127.3.61.190:44027
I20260811 19:16:44.806491 2214 heartbeater.cc:464] Registering TS with master...
I20260811 19:16:44.807286 2214 heartbeater.cc:511] Master 127.3.61.190:44027 requested a full tablet report, sending...
I20260811 19:16:44.808933 1681 ts_manager.cc:194] Registered new tserver with Master: 1562ddf0a7b94143a43c7bc580388375 (127.3.61.130:40981)
I20260811 19:16:44.809859 1681 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.130@KRBTEST.COM'} at 127.3.61.130:57953
WARNING: no policy specified for kudu/127.3.61.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab.
Entry for principal kudu/127.3.61.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab.
I20260811 19:16:44.859112 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.131:0
--local_ip_for_outbound_sockets=127.3.61.131
--webserver_interface=127.3.61.131
--webserver_port=0
--tserver_master_addrs=127.3.61.190:44027
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab
--principal=kudu/127.3.61.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:35339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:16:45.183959 1998 heartbeater.cc:503] Master 127.3.61.190:44027 was elected leader, sending a full tablet report...
W20260811 19:16:45.233363 2222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:16:45.233718 2222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:16:45.233850 2222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:16:45.241561 2222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:16:45.241741 2222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:16:45.241899 2222 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.131
I20260811 19:16:45.252404 2222 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:35339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab
--principal=kudu/127.3.61.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.61.131
--webserver_port=0
--tserver_master_addrs=127.3.61.190:44027
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.131
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:16:45.253944 2222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:16:45.255834 2222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:16:45.272903 2228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:16:45.273022 2231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:16:45.273058 2229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:16:45.274057 2222 server_base.cc:1034] running on GCE node
I20260811 19:16:45.274909 2222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:16:45.276038 2222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:16:45.277288 2222 hybrid_clock.cc:648] HybridClock initialized: now 1786475805277223 us; error 58 us; skew 500 ppm
I20260811 19:16:45.277688 2222 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:16:45 dist-test-slave-t7g5 krb5kdc[1613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475805, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:16:45.284649 2222 init.cc:377] Logged in from keytab as kudu/127.3.61.131@KRBTEST.COM (short username kudu)
I20260811 19:16:45.286463 2222 webserver.cc:460] Webserver started at http://127.3.61.131:45283/ using document root <none> and password file <none>
I20260811 19:16:45.287405 2222 fs_manager.cc:359] Metadata directory not provided
I20260811 19:16:45.287559 2222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:16:45.287930 2222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:16:45.291538 2222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/ts-2/data/instance:
uuid: "fc1632d526984baeb83b2e372ef2ad3d"
format_stamp: "Formatted at 2026-08-11 19:16:45 on dist-test-slave-t7g5"
server_key: "a53813b03f578e3bda7f646d6065f3c8"
server_key_iv: "e79b3daf7afe9d85ec5f732ab8f40d6c"
server_key_version: "encryptionkey@0"
I20260811 19:16:45.292430 2222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/ts-2/wal/instance:
uuid: "fc1632d526984baeb83b2e372ef2ad3d"
format_stamp: "Formatted at 2026-08-11 19:16:45 on dist-test-slave-t7g5"
server_key: "a53813b03f578e3bda7f646d6065f3c8"
server_key_iv: "e79b3daf7afe9d85ec5f732ab8f40d6c"
server_key_version: "encryptionkey@0"
I20260811 19:16:45.298770 2222 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260811 19:16:45.302824 2244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:45.305248 2222 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260811 19:16:45.305451 2222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/ts-2/wal
uuid: "fc1632d526984baeb83b2e372ef2ad3d"
format_stamp: "Formatted at 2026-08-11 19:16:45 on dist-test-slave-t7g5"
server_key: "a53813b03f578e3bda7f646d6065f3c8"
server_key_iv: "e79b3daf7afe9d85ec5f732ab8f40d6c"
server_key_version: "encryptionkey@0"
I20260811 19:16:45.305702 2222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:16:45.329591 2222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:16:45.335767 2222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:16:45.336114 2222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:16:45.337291 2222 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:16:45.338879 2222 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:16:45.338991 2222 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:45.339116 2222 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:16:45.339171 2222 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:45.373994 2222 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.131:36427
I20260811 19:16:45.374060 2444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.131:36427 every 8 connection(s)
I20260811 19:16:45.375362 2222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
I20260811 19:16:45.378376 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 2222
I20260811 19:16:45.378588 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/ts-2/wal/instance
I20260811 19:16:45.379196 3318 external_mini_cluster.cc:1428] Setting key 8f12399a157da411f0554e474a4fd9e2
Aug 11 19:16:45 dist-test-slave-t7g5 krb5kdc[1613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475805, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.131@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:16:45.399083 2447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:45.378265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:33415 (local address 127.3.61.190:44027)
0811 19:16:45.378788 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:45.378795 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:45.380087 (+ 1292us) server_negotiation.cc:407] Connection header received
0811 19:16:45.382316 (+ 2229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:45.382322 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:45.382428 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:45.382523 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:16:45.384095 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:45.384883 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:45.386479 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:45.386882 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:45.386992 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:45.391656 (+ 4664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:16:45.391663 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:16:45.391666 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:16:45.391717 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:16:45.395130 (+ 3413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:16:45.396497 (+ 1367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:16:45.396502 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:16:45.396504 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:16:45.396643 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:16:45.397110 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:16:45.397114 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:16:45.397116 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:16:45.397420 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:16:45.397573 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:45.398439 (+ 866us) server_negotiation.cc:299] Negotiation successful
0811 19:16:45.398682 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":290,"threads_started":1}
I20260811 19:16:45.400188 2448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:45.378986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:44027 (local address 127.3.61.131:33415)
0811 19:16:45.379919 (+ 933us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:45.379973 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 19:16:45.382047 (+ 2074us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:16:45.382775 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:16:45.382784 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:16:45.383331 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:16:45.383918 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:45.383942 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:45.385039 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:45.385043 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:45.386334 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:45.386348 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:45.387108 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:45.387116 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:45.387352 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:45.388279 (+ 927us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:16:45.388330 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:16:45.391356 (+ 3026us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:16:45.395384 (+ 4028us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:16:45.395396 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:16:45.395418 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:16:45.396342 (+ 924us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:16:45.396764 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:16:45.396767 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:16:45.396770 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:16:45.396985 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:16:45.397556 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:16:45.397566 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:16:45.398007 (+ 441us) client_negotiation.cc:769] Sending connection context
0811 19:16:45.398287 (+ 280us) client_negotiation.cc:240] Negotiation successful
0811 19:16:45.398898 (+ 611us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":374,"threads_started":1}
I20260811 19:16:45.402457 2445 heartbeater.cc:347] Connected to a master server at 127.3.61.190:44027
I20260811 19:16:45.402801 2445 heartbeater.cc:464] Registering TS with master...
I20260811 19:16:45.403556 2445 heartbeater.cc:511] Master 127.3.61.190:44027 requested a full tablet report, sending...
I20260811 19:16:45.405222 1681 ts_manager.cc:194] Registered new tserver with Master: fc1632d526984baeb83b2e372ef2ad3d (127.3.61.131:36427)
I20260811 19:16:45.406163 1681 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.131@KRBTEST.COM'} at 127.3.61.131:33415
I20260811 19:16:45.408602 3318 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 19:16:45.429855 2447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:45.412827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32898 (local address 127.3.61.190:44027)
0811 19:16:45.412994 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:45.412998 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:45.413390 (+ 392us) server_negotiation.cc:407] Connection header received
0811 19:16:45.413516 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:45.413520 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:45.413601 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:45.413738 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:16:45.415311 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:45.416068 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:45.417500 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:45.418438 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:45.418735 (+ 297us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:45.421610 (+ 2875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:16:45.421617 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:16:45.421620 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:16:45.421719 (+ 99us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:16:45.425780 (+ 4061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:16:45.426923 (+ 1143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:16:45.426928 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:16:45.426931 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:16:45.427111 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:16:45.427664 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:16:45.427668 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:16:45.427670 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:16:45.428289 (+ 619us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:16:45.428541 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:45.429356 (+ 815us) server_negotiation.cc:299] Negotiation successful
0811 19:16:45.429550 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 19:16:45.434242 1681 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32898:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 19:16:45.441749 1681 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:16:45.469496 2467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:45.456742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.131:36427 (local address 127.0.0.1:38158)
0811 19:16:45.458767 (+ 2025us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:45.458814 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 19:16:45.459062 (+ 248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:16:45.460125 (+ 1063us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:16:45.460134 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:16:45.460164 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:16:45.462645 (+ 2481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:45.462659 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:45.464458 (+ 1799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:45.464462 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:45.466426 (+ 1964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:45.466442 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:45.468357 (+ 1915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:45.468365 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:45.468652 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:45.468717 (+ 65us) client_negotiation.cc:769] Sending connection context
0811 19:16:45.468860 (+ 143us) client_negotiation.cc:240] Negotiation successful
0811 19:16:45.469067 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1727,"mutex_wait_us":62,"thread_start_us":2445,"threads_started":1}
I20260811 19:16:45.469514 2464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:45.456887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38158 (local address 127.3.61.131:36427)
0811 19:16:45.457686 (+ 799us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:45.457696 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:45.458956 (+ 1260us) server_negotiation.cc:407] Connection header received
0811 19:16:45.459314 (+ 358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:45.459323 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:45.459593 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:45.459778 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:16:45.462954 (+ 3176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:45.464261 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:45.466624 (+ 2363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:45.467936 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:45.468015 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:45.468188 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:45.468954 (+ 766us) server_negotiation.cc:299] Negotiation successful
0811 19:16:45.469141 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":906,"threads_started":1}
I20260811 19:16:45.471179 2468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:45.457567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47246 (local address 127.3.61.130:40981)
0811 19:16:45.458192 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:45.458201 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:45.459758 (+ 1557us) server_negotiation.cc:407] Connection header received
0811 19:16:45.459866 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:45.459872 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:45.460086 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:45.460270 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:16:45.462399 (+ 2129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:45.464575 (+ 2176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:45.468577 (+ 4002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:45.469976 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:45.470087 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:45.470235 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:45.470649 (+ 414us) server_negotiation.cc:299] Negotiation successful
0811 19:16:45.470833 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":463,"thread_start_us":403,"threads_started":1}
I20260811 19:16:45.472021 2461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:45.457221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.130:40981 (local address 127.0.0.1:47246)
0811 19:16:45.459482 (+ 2261us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:45.459500 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 19:16:45.459626 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:16:45.460520 (+ 894us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:16:45.460525 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:16:45.460551 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:16:45.461805 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:45.461826 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:45.464746 (+ 2920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:45.464750 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:45.468388 (+ 3638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:45.468407 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:45.470187 (+ 1780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:45.470193 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:45.470424 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:45.470459 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 19:16:45.471466 (+ 1007us) client_negotiation.cc:240] Negotiation successful
0811 19:16:45.471609 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2143,"mutex_wait_us":35,"thread_start_us":1052,"threads_started":1}
I20260811 19:16:45.472920 2471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:45.459544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:32911 (local address 127.0.0.1:47110)
0811 19:16:45.462896 (+ 3352us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:45.462919 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 19:16:45.463287 (+ 368us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:16:45.464217 (+ 930us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:16:45.464223 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:16:45.464254 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:16:45.464688 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:45.464700 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:45.466686 (+ 1986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:45.466692 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:45.468939 (+ 2247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:45.468953 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:45.471988 (+ 3035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:45.471995 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:45.472153 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:45.472186 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 19:16:45.472264 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 19:16:45.472482 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3194,"mutex_wait_us":90,"thread_start_us":2728,"threads_started":1}
I20260811 19:16:45.474428 2472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:45.459967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47110 (local address 127.3.61.129:32911)
0811 19:16:45.462010 (+ 2043us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:45.462022 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:45.463561 (+ 1539us) server_negotiation.cc:407] Connection header received
0811 19:16:45.463697 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:45.463706 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:45.463978 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:45.464185 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:16:45.464876 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:45.466504 (+ 1628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:45.469122 (+ 2618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:45.471003 (+ 1881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:45.471135 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:45.473611 (+ 2476us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:45.473810 (+ 199us) server_negotiation.cc:299] Negotiation successful
0811 19:16:45.474064 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":935,"thread_start_us":823,"threads_started":1}
I20260811 19:16:45.474535 2324 tablet_service.cc:1467] Processing CreateTablet for tablet 5d5af10abc0b407bbca1f19f2dfc5c4d (DEFAULT_TABLE table=test-table [id=cda1519bac2b47da9ac2fb0bd8d8a210]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:16:45.476156 2324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d5af10abc0b407bbca1f19f2dfc5c4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:16:45.477187 2116 tablet_service.cc:1467] Processing CreateTablet for tablet 5d5af10abc0b407bbca1f19f2dfc5c4d (DEFAULT_TABLE table=test-table [id=cda1519bac2b47da9ac2fb0bd8d8a210]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:16:45.478636 2116 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d5af10abc0b407bbca1f19f2dfc5c4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:16:45.478895 1888 tablet_service.cc:1467] Processing CreateTablet for tablet 5d5af10abc0b407bbca1f19f2dfc5c4d (DEFAULT_TABLE table=test-table [id=cda1519bac2b47da9ac2fb0bd8d8a210]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:16:45.480306 1888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d5af10abc0b407bbca1f19f2dfc5c4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:16:45.492791 2473 tablet_bootstrap.cc:492] T 5d5af10abc0b407bbca1f19f2dfc5c4d P 1562ddf0a7b94143a43c7bc580388375: Bootstrap starting.
I20260811 19:16:45.496069 2475 tablet_bootstrap.cc:492] T 5d5af10abc0b407bbca1f19f2dfc5c4d P 090c784003f544b5871e8615fea2ff51: Bootstrap starting.
I20260811 19:16:45.497171 2473 tablet_bootstrap.cc:654] T 5d5af10abc0b407bbca1f19f2dfc5c4d P 1562ddf0a7b94143a43c7bc580388375: Neither blocks nor log segments found. Creating new log.
I20260811 19:16:45.498330 2474 tablet_bootstrap.cc:492] T 5d5af10abc0b407bbca1f19f2dfc5c4d P fc1632d526984baeb83b2e372ef2ad3d: Bootstrap starting.
I20260811 19:16:45.498773 2473 log.cc:824] T 5d5af10abc0b407bbca1f19f2dfc5c4d P 1562ddf0a7b94143a43c7bc580388375: Log is configured to *not* fsync() on all Append() calls
I20260811 19:16:45.500144 2475 tablet_bootstrap.cc:654] T 5d5af10abc0b407bbca1f19f2dfc5c4d P 090c784003f544b5871e8615fea2ff51: Neither blocks nor log segments found. Creating new log.
I20260811 19:16:45.501546 2474 tablet_bootstrap.cc:654] T 5d5af10abc0b407bbca1f19f2dfc5c4d P fc1632d526984baeb83b2e372ef2ad3d: Neither blocks nor log segments found. Creating new log.
I20260811 19:16:45.501811 2475 log.cc:824] T 5d5af10abc0b407bbca1f19f2dfc5c4d P 090c784003f544b5871e8615fea2ff51: Log is configured to *not* fsync() on all Append() calls
I20260811 19:16:45.502758 2474 log.cc:824] T 5d5af10abc0b407bbca1f19f2dfc5c4d P fc1632d526984baeb83b2e372ef2ad3d: Log is configured to *not* fsync() on all Append() calls
I20260811 19:16:45.503146 2473 tablet_bootstrap.cc:492] T 5d5af10abc0b407bbca1f19f2dfc5c4d P 1562ddf0a7b94143a43c7bc580388375: No bootstrap required, opened a new log
I20260811 19:16:45.503458 2473 ts_tablet_manager.cc:1397] T 5d5af10abc0b407bbca1f19f2dfc5c4d P 1562ddf0a7b94143a43c7bc580388375: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.007s
I20260811 19:16:45.505213 2475 tablet_bootstrap.cc:492] T 5d5af10abc0b407bbca1f19f2dfc5c4d P 090c784003f544b5871e8615fea2ff51: No bootstrap required, opened a new log
I20260811 19:16:45.505602 2475 ts_tablet_manager.cc:1397] T 5d5af10abc0b407bbca1f19f2dfc5c4d P 090c784003f544b5871e8615fea2ff51: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260811 19:16:45.506946 2474 tablet_bootstrap.cc:492] T 5d5af10abc0b407bbca1f19f2dfc5c4d P fc1632d526984baeb83b2e372ef2ad3d: No bootstrap required, opened a new log
I20260811 19:16:45.507244 2474 ts_tablet_manager.cc:1397] T 5d5af10abc0b407bbca1f19f2dfc5c4d P fc1632d526984baeb83b2e372ef2ad3d: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260811 19:16:45.511547 2473 raft_consensus.cc:348] T 5d5af10abc0b407bbca1f19f2dfc5c4d P 1562ddf0a7b94143a43c7bc580388375 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc1632d526984baeb83b2e372ef2ad3d" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36427 } } peers { permanent_uuid: "090c784003f544b5871e8615fea2ff51" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 32911 } } peers { permanent_uuid: "1562ddf0a7b94143a43c7bc580388375" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 40981 } }
I20260811 19:16:45.512566 2473 raft_consensus.cc:374] T 5d5af10abc0b407bbca1f19f2dfc5c4d P 1562ddf0a7b94143a43c7bc580388375 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:16:45.512676 2473 raft_consensus.cc:729] T 5d5af10abc0b407bbca1f19f2dfc5c4d P 1562ddf0a7b94143a43c7bc580388375 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1562ddf0a7b94143a43c7bc580388375, State: Initialized, Role: FOLLOWER
I20260811 19:16:45.513410 2473 consensus_queue.cc:260] T 5d5af10abc0b407bbca1f19f2dfc5c4d P 1562ddf0a7b94143a43c7bc580388375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc1632d526984baeb83b2e372ef2ad3d" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36427 } } peers { permanent_uuid: "090c784003f544b5871e8615fea2ff51" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 32911 } } peers { permanent_uuid: "1562ddf0a7b94143a43c7bc580388375" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 40981 } }
I20260811 19:16:45.514653 2214 heartbeater.cc:503] Master 127.3.61.190:44027 was elected leader, sending a full tablet report...
I20260811 19:16:45.515241 2473 ts_tablet_manager.cc:1428] T 5d5af10abc0b407bbca1f19f2dfc5c4d P 1562ddf0a7b94143a43c7bc580388375: Time spent starting tablet: real 0.012s user 0.009s sys 0.003s
I20260811 19:16:45.517530 2474 raft_consensus.cc:348] T 5d5af10abc0b407bbca1f19f2dfc5c4d P fc1632d526984baeb83b2e372ef2ad3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc1632d526984baeb83b2e372ef2ad3d" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36427 } } peers { permanent_uuid: "090c784003f544b5871e8615fea2ff51" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 32911 } } peers { permanent_uuid: "1562ddf0a7b94143a43c7bc580388375" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 40981 } }
I20260811 19:16:45.517601 2475 raft_consensus.cc:348] T 5d5af10abc0b407bbca1f19f2dfc5c4d P 090c784003f544b5871e8615fea2ff51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc1632d526984baeb83b2e372ef2ad3d" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36427 } } peers { permanent_uuid: "090c784003f544b5871e8615fea2ff51" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 32911 } } peers { permanent_uuid: "1562ddf0a7b94143a43c7bc580388375" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 40981 } }
I20260811 19:16:45.517966 2474 raft_consensus.cc:374] T 5d5af10abc0b407bbca1f19f2dfc5c4d P fc1632d526984baeb83b2e372ef2ad3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:16:45.518064 2474 raft_consensus.cc:729] T 5d5af10abc0b407bbca1f19f2dfc5c4d P fc1632d526984baeb83b2e372ef2ad3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc1632d526984baeb83b2e372ef2ad3d, State: Initialized, Role: FOLLOWER
I20260811 19:16:45.518097 2475 raft_consensus.cc:374] T 5d5af10abc0b407bbca1f19f2dfc5c4d P 090c784003f544b5871e8615fea2ff51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:16:45.518193 2475 raft_consensus.cc:729] T 5d5af10abc0b407bbca1f19f2dfc5c4d P 090c784003f544b5871e8615fea2ff51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 090c784003f544b5871e8615fea2ff51, State: Initialized, Role: FOLLOWER
I20260811 19:16:45.518726 2474 consensus_queue.cc:260] T 5d5af10abc0b407bbca1f19f2dfc5c4d P fc1632d526984baeb83b2e372ef2ad3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc1632d526984baeb83b2e372ef2ad3d" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36427 } } peers { permanent_uuid: "090c784003f544b5871e8615fea2ff51" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 32911 } } peers { permanent_uuid: "1562ddf0a7b94143a43c7bc580388375" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 40981 } }
I20260811 19:16:45.518901 2475 consensus_queue.cc:260] T 5d5af10abc0b407bbca1f19f2dfc5c4d P 090c784003f544b5871e8615fea2ff51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc1632d526984baeb83b2e372ef2ad3d" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36427 } } peers { permanent_uuid: "090c784003f544b5871e8615fea2ff51" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 32911 } } peers { permanent_uuid: "1562ddf0a7b94143a43c7bc580388375" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 40981 } }
I20260811 19:16:45.521996 2475 ts_tablet_manager.cc:1428] T 5d5af10abc0b407bbca1f19f2dfc5c4d P 090c784003f544b5871e8615fea2ff51: Time spent starting tablet: real 0.016s user 0.011s sys 0.004s
I20260811 19:16:45.522773 2445 heartbeater.cc:503] Master 127.3.61.190:44027 was elected leader, sending a full tablet report...
I20260811 19:16:45.523490 2474 ts_tablet_manager.cc:1428] T 5d5af10abc0b407bbca1f19f2dfc5c4d P fc1632d526984baeb83b2e372ef2ad3d: Time spent starting tablet: real 0.016s user 0.016s sys 0.000s
W20260811 19:16:45.526801 2215 tablet.cc:2369] T 5d5af10abc0b407bbca1f19f2dfc5c4d P 1562ddf0a7b94143a43c7bc580388375: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 19:16:45.627171 2446 tablet.cc:2369] T 5d5af10abc0b407bbca1f19f2dfc5c4d P fc1632d526984baeb83b2e372ef2ad3d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 19:16:45.646433 1999 tablet.cc:2369] T 5d5af10abc0b407bbca1f19f2dfc5c4d P 090c784003f544b5871e8615fea2ff51: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:16:45.773317 2490 raft_consensus.cc:482] T 5d5af10abc0b407bbca1f19f2dfc5c4d P 090c784003f544b5871e8615fea2ff51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:16:45.773757 2490 raft_consensus.cc:504] T 5d5af10abc0b407bbca1f19f2dfc5c4d P 090c784003f544b5871e8615fea2ff51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc1632d526984baeb83b2e372ef2ad3d" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36427 } } peers { permanent_uuid: "090c784003f544b5871e8615fea2ff51" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 32911 } } peers { permanent_uuid: "1562ddf0a7b94143a43c7bc580388375" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 40981 } }
I20260811 19:16:45.776027 2490 leader_election.cc:288] T 5d5af10abc0b407bbca1f19f2dfc5c4d P 090c784003f544b5871e8615fea2ff51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc1632d526984baeb83b2e372ef2ad3d (127.3.61.131:36427), 1562ddf0a7b94143a43c7bc580388375 (127.3.61.130:40981)
I20260811 19:16:45.783226 2487 raft_consensus.cc:482] T 5d5af10abc0b407bbca1f19f2dfc5c4d P 1562ddf0a7b94143a43c7bc580388375 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:16:45.783638 2487 raft_consensus.cc:504] T 5d5af10abc0b407bbca1f19f2dfc5c4d P 1562ddf0a7b94143a43c7bc580388375 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc1632d526984baeb83b2e372ef2ad3d" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36427 } } peers { permanent_uuid: "090c784003f544b5871e8615fea2ff51" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 32911 } } peers { permanent_uuid: "1562ddf0a7b94143a43c7bc580388375" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 40981 } }
I20260811 19:16:45.785082 2464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:45.776598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:43971 (local address 127.3.61.131:36427)
0811 19:16:45.776810 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:45.776816 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:45.778168 (+ 1352us) server_negotiation.cc:407] Connection header received
0811 19:16:45.778521 (+ 353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:45.778526 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:45.778606 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:45.778706 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:16:45.779358 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:45.780214 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:45.782860 (+ 2646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:45.783981 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:45.784138 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:45.784215 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:45.784689 (+ 474us) server_negotiation.cc:299] Negotiation successful
0811 19:16:45.784840 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 19:16:45.785874 2492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:45.776623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.131:36427 (local address 127.3.61.129:43971)
0811 19:16:45.778064 (+ 1441us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:45.778079 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 19:16:45.778332 (+ 253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:16:45.778735 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:16:45.778739 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:16:45.778759 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:16:45.779214 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:45.779224 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:45.780383 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:45.780388 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:45.782620 (+ 2232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:45.782638 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:45.784194 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:45.784200 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:45.784394 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:45.784494 (+ 100us) client_negotiation.cc:769] Sending connection context
0811 19:16:45.785299 (+ 805us) client_negotiation.cc:240] Negotiation successful
0811 19:16:45.785450 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1330,"mutex_wait_us":24,"thread_start_us":393,"threads_started":1}
I20260811 19:16:45.786079 2468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:45.776679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:50787 (local address 127.3.61.130:40981)
0811 19:16:45.776892 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:45.776897 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:45.777896 (+ 999us) server_negotiation.cc:407] Connection header received
0811 19:16:45.778294 (+ 398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:45.778301 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:45.778385 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:45.778473 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:16:45.779179 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:45.780606 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:45.783282 (+ 2676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:45.784859 (+ 1577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:45.785036 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:45.785120 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:45.785635 (+ 515us) server_negotiation.cc:299] Negotiation successful
0811 19:16:45.785826 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 19:16:45.786927 2493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:45.776402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.130:40981 (local address 127.3.61.129:50787)
0811 19:16:45.777736 (+ 1334us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:45.777760 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 19:16:45.778036 (+ 276us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:16:45.778526 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:16:45.778530 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:16:45.778554 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:16:45.779013 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:45.779027 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:45.780821 (+ 1794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:45.780835 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:45.783079 (+ 2244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:45.783093 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:45.785269 (+ 2176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:45.785276 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:45.785451 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:45.785483 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 19:16:45.786320 (+ 837us) client_negotiation.cc:240] Negotiation successful
0811 19:16:45.786565 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":834,"mutex_wait_us":50,"thread_start_us":1079,"threads_started":1}
I20260811 19:16:45.787046 2357 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d5af10abc0b407bbca1f19f2dfc5c4d" candidate_uuid: "090c784003f544b5871e8615fea2ff51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc1632d526984baeb83b2e372ef2ad3d" is_pre_election: true
I20260811 19:16:45.787402 2487 leader_election.cc:288] T 5d5af10abc0b407bbca1f19f2dfc5c4d P 1562ddf0a7b94143a43c7bc580388375 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc1632d526984baeb83b2e372ef2ad3d (127.3.61.131:36427), 090c784003f544b5871e8615fea2ff51 (127.3.61.129:32911)
I20260811 19:16:45.787756 2357 raft_consensus.cc:2454] T 5d5af10abc0b407bbca1f19f2dfc5c4d P fc1632d526984baeb83b2e372ef2ad3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 090c784003f544b5871e8615fea2ff51 in term 0.
I20260811 19:16:45.788082 2146 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d5af10abc0b407bbca1f19f2dfc5c4d" candidate_uuid: "090c784003f544b5871e8615fea2ff51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1562ddf0a7b94143a43c7bc580388375" is_pre_election: true
I20260811 19:16:45.788635 2146 raft_consensus.cc:2454] T 5d5af10abc0b407bbca1f19f2dfc5c4d P 1562ddf0a7b94143a43c7bc580388375 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 090c784003f544b5871e8615fea2ff51 in term 0.
I20260811 19:16:45.788615 1816 leader_election.cc:302] T 5d5af10abc0b407bbca1f19f2dfc5c4d P 090c784003f544b5871e8615fea2ff51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 090c784003f544b5871e8615fea2ff51, fc1632d526984baeb83b2e372ef2ad3d; no voters:
I20260811 19:16:45.789109 2490 raft_consensus.cc:2781] T 5d5af10abc0b407bbca1f19f2dfc5c4d P 090c784003f544b5871e8615fea2ff51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:16:45.789290 2490 raft_consensus.cc:482] T 5d5af10abc0b407bbca1f19f2dfc5c4d P 090c784003f544b5871e8615fea2ff51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:16:45.789399 2490 raft_consensus.cc:3037] T 5d5af10abc0b407bbca1f19f2dfc5c4d P 090c784003f544b5871e8615fea2ff51 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:16:45.792722 2490 raft_consensus.cc:504] T 5d5af10abc0b407bbca1f19f2dfc5c4d P 090c784003f544b5871e8615fea2ff51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc1632d526984baeb83b2e372ef2ad3d" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36427 } } peers { permanent_uuid: "090c784003f544b5871e8615fea2ff51" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 32911 } } peers { permanent_uuid: "1562ddf0a7b94143a43c7bc580388375" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 40981 } }
I20260811 19:16:45.794054 2357 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d5af10abc0b407bbca1f19f2dfc5c4d" candidate_uuid: "090c784003f544b5871e8615fea2ff51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc1632d526984baeb83b2e372ef2ad3d"
I20260811 19:16:45.794312 2357 raft_consensus.cc:3037] T 5d5af10abc0b407bbca1f19f2dfc5c4d P fc1632d526984baeb83b2e372ef2ad3d [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:16:45.795302 2146 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d5af10abc0b407bbca1f19f2dfc5c4d" candidate_uuid: "090c784003f544b5871e8615fea2ff51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1562ddf0a7b94143a43c7bc580388375"
I20260811 19:16:45.795547 2146 raft_consensus.cc:3037] T 5d5af10abc0b407bbca1f19f2dfc5c4d P 1562ddf0a7b94143a43c7bc580388375 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:16:45.796840 2464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:45.787822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:42077 (local address 127.3.61.131:36427)
0811 19:16:45.787973 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:45.787978 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:45.788472 (+ 494us) server_negotiation.cc:407] Connection header received
0811 19:16:45.788832 (+ 360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:45.788837 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:45.788923 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:45.788988 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:16:45.789990 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:45.791212 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:45.794250 (+ 3038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:45.795530 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:45.795671 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:45.795748 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:45.796412 (+ 664us) server_negotiation.cc:299] Negotiation successful
0811 19:16:45.796555 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260811 19:16:45.797758 2357 raft_consensus.cc:2454] T 5d5af10abc0b407bbca1f19f2dfc5c4d P fc1632d526984baeb83b2e372ef2ad3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 090c784003f544b5871e8615fea2ff51 in term 1.
I20260811 19:16:45.798086 2494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:45.787638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.131:36427 (local address 127.3.61.130:42077)
0811 19:16:45.788383 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:45.788401 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 19:16:45.788647 (+ 246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:16:45.789227 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:16:45.789231 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:16:45.789259 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:16:45.789816 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:45.789831 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:45.791419 (+ 1588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:45.791425 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:45.794054 (+ 2629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:45.794069 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:45.796016 (+ 1947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:45.796022 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:45.796211 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:45.796244 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 19:16:45.797125 (+ 881us) client_negotiation.cc:240] Negotiation successful
0811 19:16:45.797515 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":613,"mutex_wait_us":181,"thread_start_us":417,"threads_started":1}
I20260811 19:16:45.798218 1816 leader_election.cc:302] T 5d5af10abc0b407bbca1f19f2dfc5c4d P 090c784003f544b5871e8615fea2ff51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 090c784003f544b5871e8615fea2ff51, fc1632d526984baeb83b2e372ef2ad3d; no voters:
I20260811 19:16:45.798935 2355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d5af10abc0b407bbca1f19f2dfc5c4d" candidate_uuid: "1562ddf0a7b94143a43c7bc580388375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc1632d526984baeb83b2e372ef2ad3d" is_pre_election: true
I20260811 19:16:45.799388 2355 raft_consensus.cc:2379] T 5d5af10abc0b407bbca1f19f2dfc5c4d P fc1632d526984baeb83b2e372ef2ad3d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1562ddf0a7b94143a43c7bc580388375 in current term 1: Already voted for candidate 090c784003f544b5871e8615fea2ff51 in this term.
I20260811 19:16:45.799337 2497 raft_consensus.cc:2781] T 5d5af10abc0b407bbca1f19f2dfc5c4d P 090c784003f544b5871e8615fea2ff51 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:16:45.799561 2146 raft_consensus.cc:2454] T 5d5af10abc0b407bbca1f19f2dfc5c4d P 1562ddf0a7b94143a43c7bc580388375 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 090c784003f544b5871e8615fea2ff51 in term 1.
I20260811 19:16:45.800280 2490 leader_election.cc:288] T 5d5af10abc0b407bbca1f19f2dfc5c4d P 090c784003f544b5871e8615fea2ff51 [CANDIDATE]: Term 1 election: Requested vote from peers fc1632d526984baeb83b2e372ef2ad3d (127.3.61.131:36427), 1562ddf0a7b94143a43c7bc580388375 (127.3.61.130:40981)
I20260811 19:16:45.800949 2497 raft_consensus.cc:686] T 5d5af10abc0b407bbca1f19f2dfc5c4d P 090c784003f544b5871e8615fea2ff51 [term 1 LEADER]: Becoming Leader. State: Replica: 090c784003f544b5871e8615fea2ff51, State: Running, Role: LEADER
I20260811 19:16:45.801599 2497 consensus_queue.cc:237] T 5d5af10abc0b407bbca1f19f2dfc5c4d P 090c784003f544b5871e8615fea2ff51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc1632d526984baeb83b2e372ef2ad3d" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36427 } } peers { permanent_uuid: "090c784003f544b5871e8615fea2ff51" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 32911 } } peers { permanent_uuid: "1562ddf0a7b94143a43c7bc580388375" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 40981 } }
I20260811 19:16:45.802239 2472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:45.789111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:48377 (local address 127.3.61.129:32911)
0811 19:16:45.789304 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:45.789309 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:45.794042 (+ 4733us) server_negotiation.cc:407] Connection header received
0811 19:16:45.794297 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:45.794303 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:45.794379 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:45.794439 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:16:45.795310 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:45.796321 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:45.799355 (+ 3034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:45.800819 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:45.801011 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:45.801107 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:45.801821 (+ 714us) server_negotiation.cc:299] Negotiation successful
0811 19:16:45.801992 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 19:16:45.802290 2496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:45.788562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:32911 (local address 127.3.61.130:48377)
0811 19:16:45.793950 (+ 5388us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:45.793969 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 19:16:45.794147 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:16:45.794638 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:16:45.794642 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:16:45.794669 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:16:45.795119 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:45.795132 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:45.796503 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:45.796508 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:45.798965 (+ 2457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:45.798996 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:45.801454 (+ 2458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:45.801459 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:45.801596 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:45.801621 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 19:16:45.801753 (+ 132us) client_negotiation.cc:240] Negotiation successful
0811 19:16:45.801915 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5224,"mutex_wait_us":1,"thread_start_us":4980,"threads_started":1}
I20260811 19:16:45.803210 1928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d5af10abc0b407bbca1f19f2dfc5c4d" candidate_uuid: "1562ddf0a7b94143a43c7bc580388375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "090c784003f544b5871e8615fea2ff51" is_pre_election: true
I20260811 19:16:45.804394 2032 leader_election.cc:302] T 5d5af10abc0b407bbca1f19f2dfc5c4d P 1562ddf0a7b94143a43c7bc580388375 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1562ddf0a7b94143a43c7bc580388375; no voters: 090c784003f544b5871e8615fea2ff51, fc1632d526984baeb83b2e372ef2ad3d
I20260811 19:16:45.804924 2487 raft_consensus.cc:2726] T 5d5af10abc0b407bbca1f19f2dfc5c4d P 1562ddf0a7b94143a43c7bc580388375 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 19:16:45.807605 1681 catalog_manager.cc:5495] T 5d5af10abc0b407bbca1f19f2dfc5c4d P 090c784003f544b5871e8615fea2ff51 reported cstate change: term changed from 0 to 1, leader changed from <none> to 090c784003f544b5871e8615fea2ff51 (127.3.61.129). New cstate: current_term: 1 leader_uuid: "090c784003f544b5871e8615fea2ff51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc1632d526984baeb83b2e372ef2ad3d" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "090c784003f544b5871e8615fea2ff51" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 32911 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1562ddf0a7b94143a43c7bc580388375" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 40981 } health_report { overall_health: UNKNOWN } } }
I20260811 19:16:45.847874 2472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:45.841000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47118 (local address 127.3.61.129:32911)
0811 19:16:45.841341 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:45.841349 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:45.841486 (+ 137us) server_negotiation.cc:407] Connection header received
0811 19:16:45.841901 (+ 415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:45.841906 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:45.842008 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:45.842149 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:16:45.842583 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:45.843451 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:45.844922 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:45.845441 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:45.845576 (+ 135us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:45.846388 (+ 812us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:16:45.846937 (+ 549us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:16:45.847105 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:45.847297 (+ 192us) server_negotiation.cc:299] Negotiation successful
0811 19:16:45.847456 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134}
I20260811 19:16:45.857354 2355 raft_consensus.cc:1261] T 5d5af10abc0b407bbca1f19f2dfc5c4d P fc1632d526984baeb83b2e372ef2ad3d [term 1 FOLLOWER]: Refusing update from remote peer 090c784003f544b5871e8615fea2ff51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:16:45.857385 2146 raft_consensus.cc:1261] T 5d5af10abc0b407bbca1f19f2dfc5c4d P 1562ddf0a7b94143a43c7bc580388375 [term 1 FOLLOWER]: Refusing update from remote peer 090c784003f544b5871e8615fea2ff51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:16:45.858644 2497 consensus_queue.cc:1035] T 5d5af10abc0b407bbca1f19f2dfc5c4d P 090c784003f544b5871e8615fea2ff51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1562ddf0a7b94143a43c7bc580388375" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 40981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:16:45.859107 2498 consensus_queue.cc:1035] T 5d5af10abc0b407bbca1f19f2dfc5c4d P 090c784003f544b5871e8615fea2ff51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc1632d526984baeb83b2e372ef2ad3d" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 36427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:16:45.892061 1681 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32898:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:16:45.892331 1681 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32898:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:16:45.896152 1681 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9a5aecaa6f75402586c7137edebc6757: Sending DeleteTablet for 3 replicas of tablet 5d5af10abc0b407bbca1f19f2dfc5c4d
I20260811 19:16:45.897511 2116 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d5af10abc0b407bbca1f19f2dfc5c4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:16:45 UTC) from {username='kudu', principal='kudu/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:47246
I20260811 19:16:45.897686 2324 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d5af10abc0b407bbca1f19f2dfc5c4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:16:45 UTC) from {username='kudu', principal='kudu/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:38158
I20260811 19:16:45.898541 2517 tablet_replica.cc:330] T 5d5af10abc0b407bbca1f19f2dfc5c4d P fc1632d526984baeb83b2e372ef2ad3d: stopping tablet replica
I20260811 19:16:45.898663 1888 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d5af10abc0b407bbca1f19f2dfc5c4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:16:45 UTC) from {username='kudu', principal='kudu/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:47110
I20260811 19:16:45.899170 2517 raft_consensus.cc:2229] T 5d5af10abc0b407bbca1f19f2dfc5c4d P fc1632d526984baeb83b2e372ef2ad3d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:16:45.899760 2517 raft_consensus.cc:2258] T 5d5af10abc0b407bbca1f19f2dfc5c4d P fc1632d526984baeb83b2e372ef2ad3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:16:45.899864 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 1778
I20260811 19:16:45.904147 2519 tablet_replica.cc:330] T 5d5af10abc0b407bbca1f19f2dfc5c4d P 090c784003f544b5871e8615fea2ff51: stopping tablet replica
I20260811 19:16:45.905141 2519 raft_consensus.cc:2229] T 5d5af10abc0b407bbca1f19f2dfc5c4d P 090c784003f544b5871e8615fea2ff51 [term 1 LEADER]: Raft consensus shutting down.
I20260811 19:16:45.906132 2519 raft_consensus.cc:2258] T 5d5af10abc0b407bbca1f19f2dfc5c4d P 090c784003f544b5871e8615fea2ff51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:16:45.908869 2519 ts_tablet_manager.cc:1905] T 5d5af10abc0b407bbca1f19f2dfc5c4d P 090c784003f544b5871e8615fea2ff51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:16:45.908864 2517 ts_tablet_manager.cc:1905] T 5d5af10abc0b407bbca1f19f2dfc5c4d P fc1632d526984baeb83b2e372ef2ad3d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:16:45.910121 2518 tablet_replica.cc:330] T 5d5af10abc0b407bbca1f19f2dfc5c4d P 1562ddf0a7b94143a43c7bc580388375: stopping tablet replica
I20260811 19:16:45.910591 2518 raft_consensus.cc:2229] T 5d5af10abc0b407bbca1f19f2dfc5c4d P 1562ddf0a7b94143a43c7bc580388375 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:16:45.911162 2518 raft_consensus.cc:2258] T 5d5af10abc0b407bbca1f19f2dfc5c4d P 1562ddf0a7b94143a43c7bc580388375 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:16:45.917363 2517 ts_tablet_manager.cc:1918] T 5d5af10abc0b407bbca1f19f2dfc5c4d P fc1632d526984baeb83b2e372ef2ad3d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:16:45.917672 2517 log.cc:1196] T 5d5af10abc0b407bbca1f19f2dfc5c4d P fc1632d526984baeb83b2e372ef2ad3d: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/ts-2/wal/wals/5d5af10abc0b407bbca1f19f2dfc5c4d
Aug 11 19:16:45 dist-test-slave-t7g5 krb5kdc[1613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475802, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.129@KRBTEST.COM
I20260811 19:16:45.918448 2517 ts_tablet_manager.cc:1939] T 5d5af10abc0b407bbca1f19f2dfc5c4d P fc1632d526984baeb83b2e372ef2ad3d: Deleting consensus metadata
I20260811 19:16:45.918716 2519 ts_tablet_manager.cc:1918] T 5d5af10abc0b407bbca1f19f2dfc5c4d P 090c784003f544b5871e8615fea2ff51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:16:45.918910 2519 log.cc:1196] T 5d5af10abc0b407bbca1f19f2dfc5c4d P 090c784003f544b5871e8615fea2ff51: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/ts-0/wal/wals/5d5af10abc0b407bbca1f19f2dfc5c4d
I20260811 19:16:45.919440 2519 ts_tablet_manager.cc:1939] T 5d5af10abc0b407bbca1f19f2dfc5c4d P 090c784003f544b5871e8615fea2ff51: Deleting consensus metadata
I20260811 19:16:45.919840 2518 ts_tablet_manager.cc:1905] T 5d5af10abc0b407bbca1f19f2dfc5c4d P 1562ddf0a7b94143a43c7bc580388375: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:16:45.921186 1657 catalog_manager.cc:4841] TS 090c784003f544b5871e8615fea2ff51 (127.3.61.129:32911): tablet 5d5af10abc0b407bbca1f19f2dfc5c4d (table test-table [id=cda1519bac2b47da9ac2fb0bd8d8a210]) successfully deleted
I20260811 19:16:45.921551 1657 catalog_manager.cc:4841] TS fc1632d526984baeb83b2e372ef2ad3d (127.3.61.131:36427): tablet 5d5af10abc0b407bbca1f19f2dfc5c4d (table test-table [id=cda1519bac2b47da9ac2fb0bd8d8a210]) successfully deleted
I20260811 19:16:45.926913 2518 ts_tablet_manager.cc:1918] T 5d5af10abc0b407bbca1f19f2dfc5c4d P 1562ddf0a7b94143a43c7bc580388375: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:16:45.927112 2518 log.cc:1196] T 5d5af10abc0b407bbca1f19f2dfc5c4d P 1562ddf0a7b94143a43c7bc580388375: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786475454804547-3318-0/minicluster-data/ts-1/wal/wals/5d5af10abc0b407bbca1f19f2dfc5c4d
I20260811 19:16:45.927707 2518 ts_tablet_manager.cc:1939] T 5d5af10abc0b407bbca1f19f2dfc5c4d P 1562ddf0a7b94143a43c7bc580388375: Deleting consensus metadata
I20260811 19:16:45.927747 2472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:45.901253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47124 (local address 127.3.61.129:32911)
0811 19:16:45.901479 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:45.901485 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:45.907003 (+ 5518us) server_negotiation.cc:407] Connection header received
0811 19:16:45.907322 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:45.907329 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:45.907439 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:45.907516 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:16:45.909348 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:45.910865 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:45.912759 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:45.913477 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:45.913596 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:45.919653 (+ 6057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:16:45.919668 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:16:45.919686 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:16:45.919741 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:16:45.924053 (+ 4312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:16:45.925082 (+ 1029us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:16:45.925091 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:16:45.925094 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:16:45.925252 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:16:45.925717 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:16:45.925721 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:16:45.925723 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:16:45.926228 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:16:45.926602 (+ 374us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:45.926938 (+ 336us) server_negotiation.cc:299] Negotiation successful
0811 19:16:45.927244 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260811 19:16:45.928627 1993 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:16:45.930186 1652 catalog_manager.cc:4841] TS 1562ddf0a7b94143a43c7bc580388375 (127.3.61.130:40981): tablet 5d5af10abc0b407bbca1f19f2dfc5c4d (table test-table [id=cda1519bac2b47da9ac2fb0bd8d8a210]) successfully deleted
I20260811 19:16:46.036170 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 1778
I20260811 19:16:46.065855 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 2008
Aug 11 19:16:46 dist-test-slave-t7g5 krb5kdc[1613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475802, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.130@KRBTEST.COM
I20260811 19:16:46.084313 2468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:46.067098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47262 (local address 127.3.61.130:40981)
0811 19:16:46.067395 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:46.067402 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:46.067457 (+ 55us) server_negotiation.cc:407] Connection header received
0811 19:16:46.067838 (+ 381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:46.067846 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:46.067961 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:46.068059 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:16:46.069918 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:46.070626 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:46.071967 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:46.072381 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:46.072499 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:46.077014 (+ 4515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:16:46.077032 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:16:46.077050 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:16:46.077092 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:16:46.080303 (+ 3211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:16:46.081196 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:16:46.081224 (+ 28us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:16:46.081226 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:16:46.081546 (+ 320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:16:46.082354 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:16:46.082359 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:16:46.082362 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:16:46.082895 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:16:46.083264 (+ 369us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:46.083633 (+ 369us) server_negotiation.cc:299] Negotiation successful
0811 19:16:46.083889 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
I20260811 19:16:46.085093 2201 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:16:46.181879 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 2008
I20260811 19:16:46.212750 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 2222
Aug 11 19:16:46 dist-test-slave-t7g5 krb5kdc[1613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475802, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.131@KRBTEST.COM
I20260811 19:16:46.232285 2464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:46.214130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38160 (local address 127.3.61.131:36427)
0811 19:16:46.214347 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:46.214352 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:46.214375 (+ 23us) server_negotiation.cc:407] Connection header received
0811 19:16:46.214581 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:46.214587 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:46.214667 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:46.214756 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:16:46.216306 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:46.217011 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:46.218457 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:46.218903 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:46.219019 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:46.225276 (+ 6257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:16:46.225303 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:16:46.225327 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:16:46.225383 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:16:46.228973 (+ 3590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:16:46.230036 (+ 1063us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:16:46.230044 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:16:46.230047 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:16:46.230205 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:16:46.230620 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:16:46.230624 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:16:46.230626 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:16:46.231047 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:16:46.231384 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:46.231718 (+ 334us) server_negotiation.cc:299] Negotiation successful
0811 19:16:46.231958 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 19:16:46.233218 2432 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:16:46.328827 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 2222
I20260811 19:16:46.358886 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 1634
I20260811 19:16:46.360519 1727 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:16:46.480827 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 1634
2026-08-11T19:16:46Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (5324 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-11T19:16:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:16:46Z Disabled control of system clock
I20260811 19:16:46.599386 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:35363
--webserver_interface=127.3.61.190
--webserver_port=0
--builtin_ntp_servers=127.3.61.148:34767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.61.190:35363
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 19:16:47.008976 2534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:16:47.009320 2534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:16:47.009433 2534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:16:47.017045 2534 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:16:47.017179 2534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:16:47.017220 2534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:16:47.017300 2534 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:16:47.017361 2534 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:16:47.027760 2534 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:34767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.61.190:35363
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:35363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.61.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:16:47.029377 2534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:16:47.031275 2534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:16:47.045357 2548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:16:47.045430 2547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:16:47.046185 2552 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:16:47.046701 2534 server_base.cc:1034] running on GCE node
I20260811 19:16:47.047639 2534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:16:47.049253 2534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:16:47.050510 2534 hybrid_clock.cc:648] HybridClock initialized: now 1786475807050457 us; error 40 us; skew 500 ppm
I20260811 19:16:47.050995 2534 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 19:16:47.053643 2534 webserver.cc:460] Webserver started at http://127.3.61.190:40611/ using document root <none> and password file <none>
I20260811 19:16:47.054468 2534 fs_manager.cc:359] Metadata directory not provided
I20260811 19:16:47.054600 2534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:16:47.054966 2534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:16:47.058282 2534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/master-0/data/instance:
uuid: "c7b41eea3e9e462e96cf385e94418ef2"
format_stamp: "Formatted at 2026-08-11 19:16:47 on dist-test-slave-t7g5"
server_key: "122b7b9469eb4242c33f06593d822f54"
server_key_iv: "678743ed89b50672c915084a6b42464c"
server_key_version: "encryptionkey@0"
I20260811 19:16:47.059109 2534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/master-0/wal/instance:
uuid: "c7b41eea3e9e462e96cf385e94418ef2"
format_stamp: "Formatted at 2026-08-11 19:16:47 on dist-test-slave-t7g5"
server_key: "122b7b9469eb4242c33f06593d822f54"
server_key_iv: "678743ed89b50672c915084a6b42464c"
server_key_version: "encryptionkey@0"
I20260811 19:16:47.065904 2534 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 19:16:47.071017 2567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:47.072629 2534 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260811 19:16:47.072855 2534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/master-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/master-0/wal
uuid: "c7b41eea3e9e462e96cf385e94418ef2"
format_stamp: "Formatted at 2026-08-11 19:16:47 on dist-test-slave-t7g5"
server_key: "122b7b9469eb4242c33f06593d822f54"
server_key_iv: "678743ed89b50672c915084a6b42464c"
server_key_version: "encryptionkey@0"
I20260811 19:16:47.073117 2534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:16:47.096529 2534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:16:47.097465 2534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:16:47.097788 2534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:16:47.120085 2534 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.190:35363
I20260811 19:16:47.120119 2644 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.190:35363 every 8 connection(s)
I20260811 19:16:47.122259 2534 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
I20260811 19:16:47.126550 2647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:16:47.129554 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 2534
I20260811 19:16:47.129784 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/master-0/wal/instance
I20260811 19:16:47.130401 3318 external_mini_cluster.cc:1428] Setting key 380151be43c16868e9152c7317a8057e
I20260811 19:16:47.140867 2647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7b41eea3e9e462e96cf385e94418ef2: Bootstrap starting.
I20260811 19:16:47.145612 2647 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7b41eea3e9e462e96cf385e94418ef2: Neither blocks nor log segments found. Creating new log.
I20260811 19:16:47.147372 2647 log.cc:824] T 00000000000000000000000000000000 P c7b41eea3e9e462e96cf385e94418ef2: Log is configured to *not* fsync() on all Append() calls
I20260811 19:16:47.147943 2652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:47.132809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44204 (local address 127.3.61.190:35363)
0811 19:16:47.133963 (+ 1154us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:47.133976 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:47.134024 (+ 48us) server_negotiation.cc:407] Connection header received
0811 19:16:47.136522 (+ 2498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:47.136561 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:47.137335 (+ 774us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:47.137851 (+ 516us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:16:47.138509 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:47.139847 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:47.141869 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:47.142749 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:47.142983 (+ 234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:47.144550 (+ 1567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:16:47.144560 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:16:47.144580 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:16:47.145305 (+ 725us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:16:47.145426 (+ 121us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:16:47.145454 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:16:47.145590 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:47.145996 (+ 406us) server_negotiation.cc:299] Negotiation successful
0811 19:16:47.146521 (+ 525us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":649,"thread_start_us":395,"threads_started":1}
I20260811 19:16:47.151475 2647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7b41eea3e9e462e96cf385e94418ef2: No bootstrap required, opened a new log
I20260811 19:16:47.159412 2647 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7b41eea3e9e462e96cf385e94418ef2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7b41eea3e9e462e96cf385e94418ef2" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35363 } }
I20260811 19:16:47.159826 2647 raft_consensus.cc:374] T 00000000000000000000000000000000 P c7b41eea3e9e462e96cf385e94418ef2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:16:47.159924 2647 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7b41eea3e9e462e96cf385e94418ef2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7b41eea3e9e462e96cf385e94418ef2, State: Initialized, Role: FOLLOWER
I20260811 19:16:47.160475 2647 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7b41eea3e9e462e96cf385e94418ef2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7b41eea3e9e462e96cf385e94418ef2" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35363 } }
I20260811 19:16:47.160694 2647 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7b41eea3e9e462e96cf385e94418ef2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:16:47.160799 2647 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7b41eea3e9e462e96cf385e94418ef2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:16:47.160935 2647 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7b41eea3e9e462e96cf385e94418ef2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:16:47.163344 2647 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7b41eea3e9e462e96cf385e94418ef2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7b41eea3e9e462e96cf385e94418ef2" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35363 } }
I20260811 19:16:47.163939 2647 leader_election.cc:302] T 00000000000000000000000000000000 P c7b41eea3e9e462e96cf385e94418ef2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7b41eea3e9e462e96cf385e94418ef2; no voters:
I20260811 19:16:47.164676 2647 leader_election.cc:288] T 00000000000000000000000000000000 P c7b41eea3e9e462e96cf385e94418ef2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:16:47.164969 2655 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7b41eea3e9e462e96cf385e94418ef2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:16:47.165730 2655 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7b41eea3e9e462e96cf385e94418ef2 [term 1 LEADER]: Becoming Leader. State: Replica: c7b41eea3e9e462e96cf385e94418ef2, State: Running, Role: LEADER
I20260811 19:16:47.166249 2655 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7b41eea3e9e462e96cf385e94418ef2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7b41eea3e9e462e96cf385e94418ef2" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35363 } }
I20260811 19:16:47.166863 2647 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7b41eea3e9e462e96cf385e94418ef2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:16:47.169628 2658 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7b41eea3e9e462e96cf385e94418ef2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7b41eea3e9e462e96cf385e94418ef2. Latest consensus state: current_term: 1 leader_uuid: "c7b41eea3e9e462e96cf385e94418ef2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7b41eea3e9e462e96cf385e94418ef2" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35363 } } }
I20260811 19:16:47.170255 2658 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7b41eea3e9e462e96cf385e94418ef2 [sys.catalog]: This master's current role is: LEADER
I20260811 19:16:47.170511 2656 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7b41eea3e9e462e96cf385e94418ef2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c7b41eea3e9e462e96cf385e94418ef2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7b41eea3e9e462e96cf385e94418ef2" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35363 } } }
I20260811 19:16:47.172204 2656 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7b41eea3e9e462e96cf385e94418ef2 [sys.catalog]: This master's current role is: LEADER
I20260811 19:16:47.175154 2680 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:16:47.181193 2680 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:16:47.193451 2680 catalog_manager.cc:1347] Generated new cluster ID: 855272eb7da247bb990585065ee94217
I20260811 19:16:47.193580 2680 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:16:47.211760 2680 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:16:47.213244 2680 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:16:47.224525 2680 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c7b41eea3e9e462e96cf385e94418ef2: Generated new TSK 0
I20260811 19:16:47.225873 2680 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:16:47.243141 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.129:0
--local_ip_for_outbound_sockets=127.3.61.129
--webserver_interface=127.3.61.129
--webserver_port=0
--tserver_master_addrs=127.3.61.190:35363
--builtin_ntp_servers=127.3.61.148:34767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 19:16:47.634729 2699 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:16:47.635066 2699 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:16:47.635183 2699 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:16:47.642871 2699 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:16:47.642998 2699 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:16:47.643146 2699 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.129
I20260811 19:16:47.653821 2699 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:34767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.61.129
--webserver_port=0
--tserver_master_addrs=127.3.61.190:35363
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.129
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:16:47.655205 2699 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:16:47.657133 2699 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:16:47.670993 2707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:16:47.670945 2708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:16:47.671017 2710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:16:47.672235 2699 server_base.cc:1034] running on GCE node
I20260811 19:16:47.672922 2699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:16:47.674187 2699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:16:47.675573 2699 hybrid_clock.cc:648] HybridClock initialized: now 1786475807675429 us; error 41 us; skew 500 ppm
I20260811 19:16:47.675964 2699 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 19:16:47.678987 2699 webserver.cc:460] Webserver started at http://127.3.61.129:38217/ using document root <none> and password file <none>
I20260811 19:16:47.679739 2699 fs_manager.cc:359] Metadata directory not provided
I20260811 19:16:47.679857 2699 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:16:47.680193 2699 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:16:47.683255 2699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/ts-0/data/instance:
uuid: "090a3b7616af4f83b1b3f6f1d82eb380"
format_stamp: "Formatted at 2026-08-11 19:16:47 on dist-test-slave-t7g5"
server_key: "8ab6edff1da0b5ee2076bc8f4259d660"
server_key_iv: "e63d002889029c1c3273b436dec9a46c"
server_key_version: "encryptionkey@0"
I20260811 19:16:47.684072 2699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/ts-0/wal/instance:
uuid: "090a3b7616af4f83b1b3f6f1d82eb380"
format_stamp: "Formatted at 2026-08-11 19:16:47 on dist-test-slave-t7g5"
server_key: "8ab6edff1da0b5ee2076bc8f4259d660"
server_key_iv: "e63d002889029c1c3273b436dec9a46c"
server_key_version: "encryptionkey@0"
I20260811 19:16:47.689725 2699 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260811 19:16:47.693809 2725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:47.695367 2699 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:16:47.695540 2699 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/ts-0/wal
uuid: "090a3b7616af4f83b1b3f6f1d82eb380"
format_stamp: "Formatted at 2026-08-11 19:16:47 on dist-test-slave-t7g5"
server_key: "8ab6edff1da0b5ee2076bc8f4259d660"
server_key_iv: "e63d002889029c1c3273b436dec9a46c"
server_key_version: "encryptionkey@0"
I20260811 19:16:47.695844 2699 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:16:47.723155 2699 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:16:47.724043 2699 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:16:47.724361 2699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:16:47.725451 2699 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:16:47.727010 2699 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:16:47.727102 2699 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:47.727177 2699 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:16:47.727238 2699 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:47.763082 2699 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.129:33999
I20260811 19:16:47.763197 2922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.129:33999 every 8 connection(s)
I20260811 19:16:47.764482 2699 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
I20260811 19:16:47.773121 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 2699
I20260811 19:16:47.773305 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/ts-0/wal/instance
I20260811 19:16:47.773931 3318 external_mini_cluster.cc:1428] Setting key a09cc7d5378a9fc40a5c96a56873fc4a
I20260811 19:16:47.777118 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.130:0
--local_ip_for_outbound_sockets=127.3.61.130
--webserver_interface=127.3.61.130
--webserver_port=0
--tserver_master_addrs=127.3.61.190:35363
--builtin_ntp_servers=127.3.61.148:34767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 19:16:47.780926 2927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:47.767922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:46883 (local address 127.3.61.190:35363)
0811 19:16:47.768450 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:47.768457 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:47.770073 (+ 1616us) server_negotiation.cc:407] Connection header received
0811 19:16:47.772242 (+ 2169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:47.772250 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:47.772359 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:47.772482 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:16:47.773894 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:47.774883 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:47.777194 (+ 2311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:47.777843 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:47.777985 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:47.779736 (+ 1751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:16:47.779742 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:16:47.779745 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:16:47.779765 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:16:47.779817 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:16:47.779830 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:16:47.779915 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:47.780327 (+ 412us) server_negotiation.cc:299] Negotiation successful
0811 19:16:47.780628 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":225,"threads_started":1}
I20260811 19:16:47.782495 2929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:47.768564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35363 (local address 127.3.61.129:46883)
0811 19:16:47.769866 (+ 1302us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:47.769916 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 19:16:47.771882 (+ 1966us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:16:47.772753 (+ 871us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:16:47.772762 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:16:47.772873 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:16:47.773699 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:47.773727 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:47.775086 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:47.775092 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:47.776946 (+ 1854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:47.776966 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:47.778142 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:47.778149 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:47.778467 (+ 318us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:47.779295 (+ 828us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 19:16:47.779299 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:16:47.779365 (+ 66us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 19:16:47.779482 (+ 117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:16:47.779932 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:16:47.779943 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:16:47.779950 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 19:16:47.780314 (+ 364us) client_negotiation.cc:240] Negotiation successful
0811 19:16:47.780977 (+ 663us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":386,"threads_started":1}
I20260811 19:16:47.784116 2923 heartbeater.cc:347] Connected to a master server at 127.3.61.190:35363
I20260811 19:16:47.784623 2923 heartbeater.cc:464] Registering TS with master...
I20260811 19:16:47.785785 2923 heartbeater.cc:511] Master 127.3.61.190:35363 requested a full tablet report, sending...
I20260811 19:16:47.788311 2596 ts_manager.cc:194] Registered new tserver with Master: 090a3b7616af4f83b1b3f6f1d82eb380 (127.3.61.129:33999)
I20260811 19:16:47.790148 2596 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.61.129:46883
W20260811 19:16:48.146160 2930 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:16:48.146483 2930 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:16:48.146607 2930 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:16:48.155495 2930 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:16:48.155848 2930 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:16:48.156091 2930 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.130
I20260811 19:16:48.166679 2930 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:34767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.61.130
--webserver_port=0
--tserver_master_addrs=127.3.61.190:35363
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.130
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:16:48.168081 2930 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:16:48.169948 2930 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:16:48.183214 2936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:16:48.183339 2939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:16:48.183506 2935 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:16:48.184233 2930 server_base.cc:1034] running on GCE node
I20260811 19:16:48.184912 2930 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:16:48.185878 2930 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:16:48.187106 2930 hybrid_clock.cc:648] HybridClock initialized: now 1786475808187053 us; error 39 us; skew 500 ppm
I20260811 19:16:48.187441 2930 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 19:16:48.192306 2930 webserver.cc:460] Webserver started at http://127.3.61.130:44707/ using document root <none> and password file <none>
I20260811 19:16:48.193300 2930 fs_manager.cc:359] Metadata directory not provided
I20260811 19:16:48.193441 2930 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:16:48.193887 2930 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:16:48.197559 2930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/ts-1/data/instance:
uuid: "a42f5d12412943c1af9e1f31f267ed6b"
format_stamp: "Formatted at 2026-08-11 19:16:48 on dist-test-slave-t7g5"
server_key: "c28691fed011b9fb2d90f4ac7f93dab9"
server_key_iv: "75b269edf8083795128fe1ad9d938e16"
server_key_version: "encryptionkey@0"
I20260811 19:16:48.198539 2930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/ts-1/wal/instance:
uuid: "a42f5d12412943c1af9e1f31f267ed6b"
format_stamp: "Formatted at 2026-08-11 19:16:48 on dist-test-slave-t7g5"
server_key: "c28691fed011b9fb2d90f4ac7f93dab9"
server_key_iv: "75b269edf8083795128fe1ad9d938e16"
server_key_version: "encryptionkey@0"
I20260811 19:16:48.205370 2930 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260811 19:16:48.209684 2949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:48.211313 2930 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260811 19:16:48.211495 2930 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/ts-1/wal
uuid: "a42f5d12412943c1af9e1f31f267ed6b"
format_stamp: "Formatted at 2026-08-11 19:16:48 on dist-test-slave-t7g5"
server_key: "c28691fed011b9fb2d90f4ac7f93dab9"
server_key_iv: "75b269edf8083795128fe1ad9d938e16"
server_key_version: "encryptionkey@0"
I20260811 19:16:48.211730 2930 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:16:48.236250 2930 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:16:48.237166 2930 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:16:48.237473 2930 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:16:48.238579 2930 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:16:48.240306 2930 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:16:48.240402 2930 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:48.240518 2930 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:16:48.240594 2930 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:48.275334 2930 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.130:43821
I20260811 19:16:48.275439 3149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.130:43821 every 8 connection(s)
I20260811 19:16:48.277233 2930 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
I20260811 19:16:48.287159 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 2930
I20260811 19:16:48.287343 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/ts-1/wal/instance
I20260811 19:16:48.287948 3318 external_mini_cluster.cc:1428] Setting key e8acbbd4fa3b93d107bade8655b9f093
I20260811 19:16:48.290658 2927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:48.280143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:55483 (local address 127.3.61.190:35363)
0811 19:16:48.280331 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:48.280335 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:48.281815 (+ 1480us) server_negotiation.cc:407] Connection header received
0811 19:16:48.283657 (+ 1842us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:48.283662 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:48.283732 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:48.283824 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:16:48.284897 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:48.285640 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:48.287222 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:48.287707 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:48.287836 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:48.289409 (+ 1573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:16:48.289417 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:16:48.289422 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:16:48.289454 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:16:48.289517 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:16:48.289537 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:16:48.289738 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:48.290132 (+ 394us) server_negotiation.cc:299] Negotiation successful
0811 19:16:48.290259 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 19:16:48.292151 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.131:0
--local_ip_for_outbound_sockets=127.3.61.131
--webserver_interface=127.3.61.131
--webserver_port=0
--tserver_master_addrs=127.3.61.190:35363
--builtin_ntp_servers=127.3.61.148:34767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 19:16:48.292663 3158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:48.280642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35363 (local address 127.3.61.130:55483)
0811 19:16:48.281599 (+ 957us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:48.281644 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 19:16:48.283383 (+ 1739us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:16:48.284033 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:16:48.284042 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:16:48.284126 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:16:48.284725 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:48.284742 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:48.285810 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:48.285820 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:48.287069 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:48.287083 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:48.287938 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:48.287946 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:48.288206 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:48.288935 (+ 729us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 19:16:48.288939 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:16:48.288993 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 19:16:48.289109 (+ 116us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:16:48.289808 (+ 699us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:16:48.289826 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:16:48.289837 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 19:16:48.290096 (+ 259us) client_negotiation.cc:240] Negotiation successful
0811 19:16:48.290673 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":312,"threads_started":1}
I20260811 19:16:48.294543 3155 heartbeater.cc:347] Connected to a master server at 127.3.61.190:35363
I20260811 19:16:48.294867 3155 heartbeater.cc:464] Registering TS with master...
I20260811 19:16:48.295821 3155 heartbeater.cc:511] Master 127.3.61.190:35363 requested a full tablet report, sending...
I20260811 19:16:48.297501 2596 ts_manager.cc:194] Registered new tserver with Master: a42f5d12412943c1af9e1f31f267ed6b (127.3.61.130:43821)
I20260811 19:16:48.298543 2596 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.61.130:55483
W20260811 19:16:48.679057 3161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:16:48.679364 3161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:16:48.679476 3161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:16:48.687124 3161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:16:48.687242 3161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:16:48.687407 3161 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.131
I20260811 19:16:48.697609 3161 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:34767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.61.131
--webserver_port=0
--tserver_master_addrs=127.3.61.190:35363
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.131
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:16:48.699031 3161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:16:48.700975 3161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:16:48.715005 3172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:16:48.715040 3169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:16:48.715061 3167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:16:48.715854 3161 server_base.cc:1034] running on GCE node
I20260811 19:16:48.716482 3161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:16:48.717540 3161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:16:48.718801 3161 hybrid_clock.cc:648] HybridClock initialized: now 1786475808718722 us; error 58 us; skew 500 ppm
I20260811 19:16:48.719157 3161 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 19:16:48.721544 3161 webserver.cc:460] Webserver started at http://127.3.61.131:38659/ using document root <none> and password file <none>
I20260811 19:16:48.722366 3161 fs_manager.cc:359] Metadata directory not provided
I20260811 19:16:48.722476 3161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:16:48.722801 3161 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:16:48.725791 3161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/ts-2/data/instance:
uuid: "9af54edecfbf49b1be008d796de5a187"
format_stamp: "Formatted at 2026-08-11 19:16:48 on dist-test-slave-t7g5"
server_key: "75f93c2ba09fc9e9d0097b1c213bde63"
server_key_iv: "cc476ddecf132a75a8bec7d9ed9efc3c"
server_key_version: "encryptionkey@0"
I20260811 19:16:48.726570 3161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/ts-2/wal/instance:
uuid: "9af54edecfbf49b1be008d796de5a187"
format_stamp: "Formatted at 2026-08-11 19:16:48 on dist-test-slave-t7g5"
server_key: "75f93c2ba09fc9e9d0097b1c213bde63"
server_key_iv: "cc476ddecf132a75a8bec7d9ed9efc3c"
server_key_version: "encryptionkey@0"
I20260811 19:16:48.732475 3161 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 19:16:48.736932 3180 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:48.738651 3161 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 19:16:48.738827 3161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/ts-2/wal
uuid: "9af54edecfbf49b1be008d796de5a187"
format_stamp: "Formatted at 2026-08-11 19:16:48 on dist-test-slave-t7g5"
server_key: "75f93c2ba09fc9e9d0097b1c213bde63"
server_key_iv: "cc476ddecf132a75a8bec7d9ed9efc3c"
server_key_version: "encryptionkey@0"
I20260811 19:16:48.739089 3161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:16:48.766047 3161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:16:48.766957 3161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:16:48.767313 3161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:16:48.768586 3161 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:16:48.770274 3161 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:16:48.770376 3161 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:48.770463 3161 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:16:48.770539 3161 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:48.793596 2923 heartbeater.cc:503] Master 127.3.61.190:35363 was elected leader, sending a full tablet report...
I20260811 19:16:48.806360 3161 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.131:45191
I20260811 19:16:48.806428 3382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.131:45191 every 8 connection(s)
I20260811 19:16:48.807675 3161 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
I20260811 19:16:48.814154 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 3161
I20260811 19:16:48.814342 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/ts-2/wal/instance
I20260811 19:16:48.814824 3318 external_mini_cluster.cc:1428] Setting key 5fd316018ab5e3c3fa2351360b11f449
I20260811 19:16:48.821856 3387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:48.810489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:54583 (local address 127.3.61.190:35363)
0811 19:16:48.811071 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:48.811076 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:48.812825 (+ 1749us) server_negotiation.cc:407] Connection header received
0811 19:16:48.814860 (+ 2035us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:48.814866 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:48.814959 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:48.815060 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:16:48.816485 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:48.817380 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:48.818996 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:48.819404 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:48.819514 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:48.820920 (+ 1406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:16:48.820926 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:16:48.820929 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:16:48.820948 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:16:48.820996 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:16:48.821008 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:16:48.821083 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:48.821314 (+ 231us) server_negotiation.cc:299] Negotiation successful
0811 19:16:48.821422 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":294,"threads_started":1}
I20260811 19:16:48.822842 3388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:48.811446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35363 (local address 127.3.61.131:54583)
0811 19:16:48.812637 (+ 1191us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:48.812679 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 19:16:48.814558 (+ 1879us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:16:48.815345 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:16:48.815357 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:16:48.815509 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:16:48.816308 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:48.816336 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:48.817533 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:48.817537 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:48.818854 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:48.818868 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:48.819633 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:48.819642 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:48.819887 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:48.820578 (+ 691us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 19:16:48.820581 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:16:48.820625 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 19:16:48.820723 (+ 98us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:16:48.821097 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:16:48.821108 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:16:48.821116 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 19:16:48.821282 (+ 166us) client_negotiation.cc:240] Negotiation successful
0811 19:16:48.821738 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":648,"thread_start_us":380,"threads_started":1}
I20260811 19:16:48.824760 3383 heartbeater.cc:347] Connected to a master server at 127.3.61.190:35363
I20260811 19:16:48.825099 3383 heartbeater.cc:464] Registering TS with master...
I20260811 19:16:48.825956 3383 heartbeater.cc:511] Master 127.3.61.190:35363 requested a full tablet report, sending...
I20260811 19:16:48.827605 2596 ts_manager.cc:194] Registered new tserver with Master: 9af54edecfbf49b1be008d796de5a187 (127.3.61.131:45191)
I20260811 19:16:48.828408 2596 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.61.131:54583
I20260811 19:16:48.830421 3318 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 19:16:48.841224 3387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:48.834703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44218 (local address 127.3.61.190:35363)
0811 19:16:48.834933 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:48.834940 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:48.835260 (+ 320us) server_negotiation.cc:407] Connection header received
0811 19:16:48.835524 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:48.835531 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:48.835599 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:48.835694 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:16:48.836192 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:48.836894 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:48.838368 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:48.838786 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:48.838895 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:48.840143 (+ 1248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:16:48.840148 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:16:48.840151 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:16:48.840169 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:16:48.840213 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:16:48.840226 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:16:48.840342 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:48.840884 (+ 542us) server_negotiation.cc:299] Negotiation successful
0811 19:16:48.840942 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 19:16:48.846577 2596 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44218:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 19:16:48.854179 2596 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:16:48.892454 3413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:48.867876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.131:45191 (local address 127.0.0.1:53480)
0811 19:16:48.882514 (+ 14638us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:48.882537 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 19:16:48.882712 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:16:48.883932 (+ 1220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:16:48.883942 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:16:48.883980 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:16:48.885047 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:48.885066 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:48.887069 (+ 2003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:48.887074 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:48.889415 (+ 2341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:48.889439 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:48.891475 (+ 2036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:48.891482 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:48.891686 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:48.891739 (+ 53us) client_negotiation.cc:769] Sending connection context
0811 19:16:48.891863 (+ 124us) client_negotiation.cc:240] Negotiation successful
0811 19:16:48.892090 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14403,"mutex_wait_us":5,"thread_start_us":14235,"threads_started":1}
I20260811 19:16:48.892494 3416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:48.868656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53480 (local address 127.3.61.131:45191)
0811 19:16:48.881901 (+ 13245us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:48.881912 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:48.882866 (+ 954us) server_negotiation.cc:407] Connection header received
0811 19:16:48.883002 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:48.883012 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:48.883316 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:48.883544 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:16:48.885280 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:48.886891 (+ 1611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:48.889743 (+ 2852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:48.891291 (+ 1548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:48.891414 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:48.891729 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:48.891934 (+ 205us) server_negotiation.cc:299] Negotiation successful
0811 19:16:48.892107 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13036,"thread_start_us":12846,"threads_started":1}
I20260811 19:16:48.893600 3415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:48.868572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.130:43821 (local address 127.0.0.1:42574)
0811 19:16:48.884845 (+ 16273us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:48.884865 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 19:16:48.885020 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:16:48.885969 (+ 949us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:16:48.885974 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:16:48.886001 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:16:48.886492 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:48.886505 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:48.888383 (+ 1878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:48.888388 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:48.890543 (+ 2155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:48.890556 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:48.892763 (+ 2207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:48.892771 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:48.892968 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:48.893000 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 19:16:48.893091 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 19:16:48.893236 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16050,"mutex_wait_us":148,"thread_start_us":16000,"threads_started":1}
I20260811 19:16:48.893744 3418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:48.869144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42574 (local address 127.3.61.130:43821)
0811 19:16:48.884105 (+ 14961us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:48.884115 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:48.884988 (+ 873us) server_negotiation.cc:407] Connection header received
0811 19:16:48.885190 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:48.885198 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:48.885460 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:48.885694 (+ 234us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:16:48.886685 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:48.888223 (+ 1538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:48.890812 (+ 2589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:48.892226 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:48.892376 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:48.892620 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:48.893196 (+ 576us) server_negotiation.cc:299] Negotiation successful
0811 19:16:48.893379 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14783,"thread_start_us":14638,"threads_started":1}
I20260811 19:16:48.894110 3409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:48.867690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:33999 (local address 127.0.0.1:35450)
0811 19:16:48.879923 (+ 12233us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:48.879990 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0811 19:16:48.880274 (+ 284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:16:48.886071 (+ 5797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:16:48.886076 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:16:48.886110 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:16:48.887161 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:48.887173 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:48.888538 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:48.888544 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:48.890953 (+ 2409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:48.890969 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:48.893120 (+ 2151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:48.893127 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:48.893293 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:48.893324 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 19:16:48.893405 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 19:16:48.893531 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11794,"mutex_wait_us":39,"thread_start_us":12847,"threads_started":1}
I20260811 19:16:48.895136 3412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:48.867804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35450 (local address 127.3.61.129:33999)
0811 19:16:48.885446 (+ 17642us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:48.885452 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:48.885477 (+ 25us) server_negotiation.cc:407] Connection header received
0811 19:16:48.885586 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:48.885593 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:48.885844 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:48.886704 (+ 860us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:16:48.887338 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:48.888386 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:48.891172 (+ 2786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:48.892589 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:48.892725 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:48.892995 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:48.893733 (+ 738us) server_negotiation.cc:299] Negotiation successful
0811 19:16:48.893926 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17026,"thread_start_us":16863,"threads_started":1}
I20260811 19:16:48.897259 3034 tablet_service.cc:1467] Processing CreateTablet for tablet 356a2bde46834db7acfa5c2856f5c5a4 (DEFAULT_TABLE table=test-table [id=e97ac63142364dd084cf8abfffc9b2ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:16:48.898865 2811 tablet_service.cc:1467] Processing CreateTablet for tablet 356a2bde46834db7acfa5c2856f5c5a4 (DEFAULT_TABLE table=test-table [id=e97ac63142364dd084cf8abfffc9b2ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:16:48.897254 3257 tablet_service.cc:1467] Processing CreateTablet for tablet 356a2bde46834db7acfa5c2856f5c5a4 (DEFAULT_TABLE table=test-table [id=e97ac63142364dd084cf8abfffc9b2ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:16:48.900530 2811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 356a2bde46834db7acfa5c2856f5c5a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:16:48.900663 3034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 356a2bde46834db7acfa5c2856f5c5a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:16:48.901139 3257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 356a2bde46834db7acfa5c2856f5c5a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:16:48.916239 3419 tablet_bootstrap.cc:492] T 356a2bde46834db7acfa5c2856f5c5a4 P 090a3b7616af4f83b1b3f6f1d82eb380: Bootstrap starting.
I20260811 19:16:48.920842 3419 tablet_bootstrap.cc:654] T 356a2bde46834db7acfa5c2856f5c5a4 P 090a3b7616af4f83b1b3f6f1d82eb380: Neither blocks nor log segments found. Creating new log.
I20260811 19:16:48.922065 3421 tablet_bootstrap.cc:492] T 356a2bde46834db7acfa5c2856f5c5a4 P 9af54edecfbf49b1be008d796de5a187: Bootstrap starting.
I20260811 19:16:48.922433 3419 log.cc:824] T 356a2bde46834db7acfa5c2856f5c5a4 P 090a3b7616af4f83b1b3f6f1d82eb380: Log is configured to *not* fsync() on all Append() calls
I20260811 19:16:48.924968 3423 tablet_bootstrap.cc:492] T 356a2bde46834db7acfa5c2856f5c5a4 P a42f5d12412943c1af9e1f31f267ed6b: Bootstrap starting.
I20260811 19:16:48.926317 3421 tablet_bootstrap.cc:654] T 356a2bde46834db7acfa5c2856f5c5a4 P 9af54edecfbf49b1be008d796de5a187: Neither blocks nor log segments found. Creating new log.
I20260811 19:16:48.927356 3421 log.cc:824] T 356a2bde46834db7acfa5c2856f5c5a4 P 9af54edecfbf49b1be008d796de5a187: Log is configured to *not* fsync() on all Append() calls
I20260811 19:16:48.927567 3419 tablet_bootstrap.cc:492] T 356a2bde46834db7acfa5c2856f5c5a4 P 090a3b7616af4f83b1b3f6f1d82eb380: No bootstrap required, opened a new log
I20260811 19:16:48.927904 3419 ts_tablet_manager.cc:1397] T 356a2bde46834db7acfa5c2856f5c5a4 P 090a3b7616af4f83b1b3f6f1d82eb380: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260811 19:16:48.928700 3423 tablet_bootstrap.cc:654] T 356a2bde46834db7acfa5c2856f5c5a4 P a42f5d12412943c1af9e1f31f267ed6b: Neither blocks nor log segments found. Creating new log.
I20260811 19:16:48.930022 3423 log.cc:824] T 356a2bde46834db7acfa5c2856f5c5a4 P a42f5d12412943c1af9e1f31f267ed6b: Log is configured to *not* fsync() on all Append() calls
I20260811 19:16:48.930312 3421 tablet_bootstrap.cc:492] T 356a2bde46834db7acfa5c2856f5c5a4 P 9af54edecfbf49b1be008d796de5a187: No bootstrap required, opened a new log
I20260811 19:16:48.930579 3421 ts_tablet_manager.cc:1397] T 356a2bde46834db7acfa5c2856f5c5a4 P 9af54edecfbf49b1be008d796de5a187: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 19:16:48.933810 3423 tablet_bootstrap.cc:492] T 356a2bde46834db7acfa5c2856f5c5a4 P a42f5d12412943c1af9e1f31f267ed6b: No bootstrap required, opened a new log
I20260811 19:16:48.934156 3423 ts_tablet_manager.cc:1397] T 356a2bde46834db7acfa5c2856f5c5a4 P a42f5d12412943c1af9e1f31f267ed6b: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260811 19:16:48.938867 3419 raft_consensus.cc:348] T 356a2bde46834db7acfa5c2856f5c5a4 P 090a3b7616af4f83b1b3f6f1d82eb380 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "090a3b7616af4f83b1b3f6f1d82eb380" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 33999 } } peers { permanent_uuid: "9af54edecfbf49b1be008d796de5a187" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 45191 } } peers { permanent_uuid: "a42f5d12412943c1af9e1f31f267ed6b" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 43821 } }
I20260811 19:16:48.939458 3419 raft_consensus.cc:374] T 356a2bde46834db7acfa5c2856f5c5a4 P 090a3b7616af4f83b1b3f6f1d82eb380 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:16:48.939564 3419 raft_consensus.cc:729] T 356a2bde46834db7acfa5c2856f5c5a4 P 090a3b7616af4f83b1b3f6f1d82eb380 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 090a3b7616af4f83b1b3f6f1d82eb380, State: Initialized, Role: FOLLOWER
I20260811 19:16:48.939262 3421 raft_consensus.cc:348] T 356a2bde46834db7acfa5c2856f5c5a4 P 9af54edecfbf49b1be008d796de5a187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "090a3b7616af4f83b1b3f6f1d82eb380" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 33999 } } peers { permanent_uuid: "9af54edecfbf49b1be008d796de5a187" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 45191 } } peers { permanent_uuid: "a42f5d12412943c1af9e1f31f267ed6b" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 43821 } }
I20260811 19:16:48.939735 3421 raft_consensus.cc:374] T 356a2bde46834db7acfa5c2856f5c5a4 P 9af54edecfbf49b1be008d796de5a187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:16:48.939852 3421 raft_consensus.cc:729] T 356a2bde46834db7acfa5c2856f5c5a4 P 9af54edecfbf49b1be008d796de5a187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9af54edecfbf49b1be008d796de5a187, State: Initialized, Role: FOLLOWER
I20260811 19:16:48.940431 3421 consensus_queue.cc:260] T 356a2bde46834db7acfa5c2856f5c5a4 P 9af54edecfbf49b1be008d796de5a187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "090a3b7616af4f83b1b3f6f1d82eb380" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 33999 } } peers { permanent_uuid: "9af54edecfbf49b1be008d796de5a187" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 45191 } } peers { permanent_uuid: "a42f5d12412943c1af9e1f31f267ed6b" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 43821 } }
I20260811 19:16:48.940404 3419 consensus_queue.cc:260] T 356a2bde46834db7acfa5c2856f5c5a4 P 090a3b7616af4f83b1b3f6f1d82eb380 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "090a3b7616af4f83b1b3f6f1d82eb380" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 33999 } } peers { permanent_uuid: "9af54edecfbf49b1be008d796de5a187" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 45191 } } peers { permanent_uuid: "a42f5d12412943c1af9e1f31f267ed6b" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 43821 } }
I20260811 19:16:48.942209 3383 heartbeater.cc:503] Master 127.3.61.190:35363 was elected leader, sending a full tablet report...
I20260811 19:16:48.942605 3419 ts_tablet_manager.cc:1428] T 356a2bde46834db7acfa5c2856f5c5a4 P 090a3b7616af4f83b1b3f6f1d82eb380: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
I20260811 19:16:48.942840 3421 ts_tablet_manager.cc:1428] T 356a2bde46834db7acfa5c2856f5c5a4 P 9af54edecfbf49b1be008d796de5a187: Time spent starting tablet: real 0.012s user 0.009s sys 0.003s
I20260811 19:16:48.945271 3423 raft_consensus.cc:348] T 356a2bde46834db7acfa5c2856f5c5a4 P a42f5d12412943c1af9e1f31f267ed6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "090a3b7616af4f83b1b3f6f1d82eb380" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 33999 } } peers { permanent_uuid: "9af54edecfbf49b1be008d796de5a187" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 45191 } } peers { permanent_uuid: "a42f5d12412943c1af9e1f31f267ed6b" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 43821 } }
I20260811 19:16:48.945701 3423 raft_consensus.cc:374] T 356a2bde46834db7acfa5c2856f5c5a4 P a42f5d12412943c1af9e1f31f267ed6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:16:48.945806 3423 raft_consensus.cc:729] T 356a2bde46834db7acfa5c2856f5c5a4 P a42f5d12412943c1af9e1f31f267ed6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a42f5d12412943c1af9e1f31f267ed6b, State: Initialized, Role: FOLLOWER
I20260811 19:16:48.946488 3423 consensus_queue.cc:260] T 356a2bde46834db7acfa5c2856f5c5a4 P a42f5d12412943c1af9e1f31f267ed6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "090a3b7616af4f83b1b3f6f1d82eb380" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 33999 } } peers { permanent_uuid: "9af54edecfbf49b1be008d796de5a187" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 45191 } } peers { permanent_uuid: "a42f5d12412943c1af9e1f31f267ed6b" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 43821 } }
I20260811 19:16:48.948529 3423 ts_tablet_manager.cc:1428] T 356a2bde46834db7acfa5c2856f5c5a4 P a42f5d12412943c1af9e1f31f267ed6b: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
I20260811 19:16:48.949216 3155 heartbeater.cc:503] Master 127.3.61.190:35363 was elected leader, sending a full tablet report...
I20260811 19:16:48.990281 3427 raft_consensus.cc:482] T 356a2bde46834db7acfa5c2856f5c5a4 P 9af54edecfbf49b1be008d796de5a187 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:16:48.990608 3427 raft_consensus.cc:504] T 356a2bde46834db7acfa5c2856f5c5a4 P 9af54edecfbf49b1be008d796de5a187 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "090a3b7616af4f83b1b3f6f1d82eb380" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 33999 } } peers { permanent_uuid: "9af54edecfbf49b1be008d796de5a187" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 45191 } } peers { permanent_uuid: "a42f5d12412943c1af9e1f31f267ed6b" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 43821 } }
I20260811 19:16:48.992326 3427 leader_election.cc:288] T 356a2bde46834db7acfa5c2856f5c5a4 P 9af54edecfbf49b1be008d796de5a187 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 090a3b7616af4f83b1b3f6f1d82eb380 (127.3.61.129:33999), a42f5d12412943c1af9e1f31f267ed6b (127.3.61.130:43821)
I20260811 19:16:48.998739 3388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:48.992665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.130:43821 (local address 127.3.61.131:34481)
0811 19:16:48.992978 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:48.993001 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 19:16:48.993138 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:16:48.993635 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:16:48.993644 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:16:48.993708 (+ 64us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:16:48.993907 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:48.993919 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:48.995025 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:48.995029 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:48.996433 (+ 1404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:48.996448 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:48.997757 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:48.997766 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:48.998002 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:48.998033 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 19:16:48.998138 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 19:16:48.998420 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186}
I20260811 19:16:48.998728 3418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:48.992901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:34481 (local address 127.3.61.130:43821)
0811 19:16:48.993116 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:48.993122 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:48.993165 (+ 43us) server_negotiation.cc:407] Connection header received
0811 19:16:48.993373 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:48.993379 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:48.993471 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:48.993576 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:16:48.994056 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:48.994850 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:48.996604 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:48.997363 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:48.997812 (+ 449us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:48.997914 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:48.998269 (+ 355us) server_negotiation.cc:299] Negotiation successful
0811 19:16:48.998448 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 19:16:48.999277 3436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:48.992872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:33999 (local address 127.3.61.131:50663)
0811 19:16:48.993684 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:48.993701 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 19:16:48.993820 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:16:48.994126 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:16:48.994131 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:16:48.994157 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:16:48.994470 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:48.994479 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:48.995483 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:48.995486 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:48.996842 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:48.996856 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:48.998520 (+ 1664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:48.998527 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:48.998710 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:48.998743 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 19:16:48.998835 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 19:16:48.998971 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":656,"thread_start_us":436,"threads_started":1}
I20260811 19:16:48.999619 3412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:48.992987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:50663 (local address 127.3.61.129:33999)
0811 19:16:48.993208 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:48.993215 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:48.993784 (+ 569us) server_negotiation.cc:407] Connection header received
0811 19:16:48.993935 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:48.993939 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:48.994004 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:48.994086 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:16:48.994632 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:48.995319 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:48.997369 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:48.998325 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:48.998517 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:48.998592 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:48.999245 (+ 653us) server_negotiation.cc:299] Negotiation successful
0811 19:16:48.999390 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260811 19:16:49.000383 3075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "356a2bde46834db7acfa5c2856f5c5a4" candidate_uuid: "9af54edecfbf49b1be008d796de5a187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a42f5d12412943c1af9e1f31f267ed6b" is_pre_election: true
I20260811 19:16:49.000586 2839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "356a2bde46834db7acfa5c2856f5c5a4" candidate_uuid: "9af54edecfbf49b1be008d796de5a187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "090a3b7616af4f83b1b3f6f1d82eb380" is_pre_election: true
I20260811 19:16:49.001140 3075 raft_consensus.cc:2454] T 356a2bde46834db7acfa5c2856f5c5a4 P a42f5d12412943c1af9e1f31f267ed6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9af54edecfbf49b1be008d796de5a187 in term 0.
I20260811 19:16:49.001215 2839 raft_consensus.cc:2454] T 356a2bde46834db7acfa5c2856f5c5a4 P 090a3b7616af4f83b1b3f6f1d82eb380 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9af54edecfbf49b1be008d796de5a187 in term 0.
I20260811 19:16:49.001976 3184 leader_election.cc:302] T 356a2bde46834db7acfa5c2856f5c5a4 P 9af54edecfbf49b1be008d796de5a187 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 090a3b7616af4f83b1b3f6f1d82eb380, 9af54edecfbf49b1be008d796de5a187, a42f5d12412943c1af9e1f31f267ed6b; no voters:
I20260811 19:16:49.002483 3427 raft_consensus.cc:2781] T 356a2bde46834db7acfa5c2856f5c5a4 P 9af54edecfbf49b1be008d796de5a187 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:16:49.002672 3427 raft_consensus.cc:482] T 356a2bde46834db7acfa5c2856f5c5a4 P 9af54edecfbf49b1be008d796de5a187 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:16:49.002779 3427 raft_consensus.cc:3037] T 356a2bde46834db7acfa5c2856f5c5a4 P 9af54edecfbf49b1be008d796de5a187 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:16:49.005479 3427 raft_consensus.cc:504] T 356a2bde46834db7acfa5c2856f5c5a4 P 9af54edecfbf49b1be008d796de5a187 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "090a3b7616af4f83b1b3f6f1d82eb380" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 33999 } } peers { permanent_uuid: "9af54edecfbf49b1be008d796de5a187" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 45191 } } peers { permanent_uuid: "a42f5d12412943c1af9e1f31f267ed6b" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 43821 } }
I20260811 19:16:49.006201 3427 leader_election.cc:288] T 356a2bde46834db7acfa5c2856f5c5a4 P 9af54edecfbf49b1be008d796de5a187 [CANDIDATE]: Term 1 election: Requested vote from peers 090a3b7616af4f83b1b3f6f1d82eb380 (127.3.61.129:33999), a42f5d12412943c1af9e1f31f267ed6b (127.3.61.130:43821)
I20260811 19:16:49.006848 2839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "356a2bde46834db7acfa5c2856f5c5a4" candidate_uuid: "9af54edecfbf49b1be008d796de5a187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "090a3b7616af4f83b1b3f6f1d82eb380"
I20260811 19:16:49.006894 3075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "356a2bde46834db7acfa5c2856f5c5a4" candidate_uuid: "9af54edecfbf49b1be008d796de5a187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a42f5d12412943c1af9e1f31f267ed6b"
I20260811 19:16:49.007068 2839 raft_consensus.cc:3037] T 356a2bde46834db7acfa5c2856f5c5a4 P 090a3b7616af4f83b1b3f6f1d82eb380 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:16:49.007160 3075 raft_consensus.cc:3037] T 356a2bde46834db7acfa5c2856f5c5a4 P a42f5d12412943c1af9e1f31f267ed6b [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:16:49.009639 2839 raft_consensus.cc:2454] T 356a2bde46834db7acfa5c2856f5c5a4 P 090a3b7616af4f83b1b3f6f1d82eb380 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9af54edecfbf49b1be008d796de5a187 in term 1.
I20260811 19:16:49.009847 3075 raft_consensus.cc:2454] T 356a2bde46834db7acfa5c2856f5c5a4 P a42f5d12412943c1af9e1f31f267ed6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9af54edecfbf49b1be008d796de5a187 in term 1.
I20260811 19:16:49.010226 3184 leader_election.cc:302] T 356a2bde46834db7acfa5c2856f5c5a4 P 9af54edecfbf49b1be008d796de5a187 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 090a3b7616af4f83b1b3f6f1d82eb380, 9af54edecfbf49b1be008d796de5a187; no voters:
I20260811 19:16:49.010588 3427 raft_consensus.cc:2781] T 356a2bde46834db7acfa5c2856f5c5a4 P 9af54edecfbf49b1be008d796de5a187 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:16:49.011059 3427 raft_consensus.cc:686] T 356a2bde46834db7acfa5c2856f5c5a4 P 9af54edecfbf49b1be008d796de5a187 [term 1 LEADER]: Becoming Leader. State: Replica: 9af54edecfbf49b1be008d796de5a187, State: Running, Role: LEADER
I20260811 19:16:49.011668 3427 consensus_queue.cc:237] T 356a2bde46834db7acfa5c2856f5c5a4 P 9af54edecfbf49b1be008d796de5a187 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "090a3b7616af4f83b1b3f6f1d82eb380" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 33999 } } peers { permanent_uuid: "9af54edecfbf49b1be008d796de5a187" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 45191 } } peers { permanent_uuid: "a42f5d12412943c1af9e1f31f267ed6b" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 43821 } }
W20260811 19:16:49.017319 2924 tablet.cc:2369] T 356a2bde46834db7acfa5c2856f5c5a4 P 090a3b7616af4f83b1b3f6f1d82eb380: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:16:49.017352 2596 catalog_manager.cc:5495] T 356a2bde46834db7acfa5c2856f5c5a4 P 9af54edecfbf49b1be008d796de5a187 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9af54edecfbf49b1be008d796de5a187 (127.3.61.131). New cstate: current_term: 1 leader_uuid: "9af54edecfbf49b1be008d796de5a187" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "090a3b7616af4f83b1b3f6f1d82eb380" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 33999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9af54edecfbf49b1be008d796de5a187" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 45191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a42f5d12412943c1af9e1f31f267ed6b" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 43821 } health_report { overall_health: UNKNOWN } } }
W20260811 19:16:49.029567 3156 tablet.cc:2369] T 356a2bde46834db7acfa5c2856f5c5a4 P a42f5d12412943c1af9e1f31f267ed6b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 19:16:49.059414 3385 tablet.cc:2369] T 356a2bde46834db7acfa5c2856f5c5a4 P 9af54edecfbf49b1be008d796de5a187: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:16:49.060325 3416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:49.053779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53496 (local address 127.3.61.131:45191)
0811 19:16:49.054032 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:49.054039 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:49.054064 (+ 25us) server_negotiation.cc:407] Connection header received
0811 19:16:49.054322 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:49.054327 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:49.054444 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:49.054619 (+ 175us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:16:49.055256 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:49.056180 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:49.057711 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:49.058133 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:49.058249 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:49.058908 (+ 659us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:16:49.059620 (+ 712us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:16:49.059807 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:49.060018 (+ 211us) server_negotiation.cc:299] Negotiation successful
0811 19:16:49.060071 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 19:16:49.069109 2839 raft_consensus.cc:1261] T 356a2bde46834db7acfa5c2856f5c5a4 P 090a3b7616af4f83b1b3f6f1d82eb380 [term 1 FOLLOWER]: Refusing update from remote peer 9af54edecfbf49b1be008d796de5a187: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:16:49.069109 3075 raft_consensus.cc:1261] T 356a2bde46834db7acfa5c2856f5c5a4 P a42f5d12412943c1af9e1f31f267ed6b [term 1 FOLLOWER]: Refusing update from remote peer 9af54edecfbf49b1be008d796de5a187: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:16:49.070921 3438 consensus_queue.cc:1035] T 356a2bde46834db7acfa5c2856f5c5a4 P 9af54edecfbf49b1be008d796de5a187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "090a3b7616af4f83b1b3f6f1d82eb380" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 33999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:16:49.071491 3427 consensus_queue.cc:1035] T 356a2bde46834db7acfa5c2856f5c5a4 P 9af54edecfbf49b1be008d796de5a187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a42f5d12412943c1af9e1f31f267ed6b" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 43821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:16:49.086174 3444 mvcc.cc:205] Tried to move back new op lower bound from 7317404913935351808 to 7317404913716670464. Current Snapshot: MvccSnapshot[applied={T|T < 7317404913935351808}]
I20260811 19:16:49.090080 3446 mvcc.cc:205] Tried to move back new op lower bound from 7317404913935351808 to 7317404913716670464. Current Snapshot: MvccSnapshot[applied={T|T < 7317404913935351808}]
I20260811 19:16:49.102455 2596 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:16:49.102767 2596 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:16:49.106976 2596 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c7b41eea3e9e462e96cf385e94418ef2: Sending DeleteTablet for 3 replicas of tablet 356a2bde46834db7acfa5c2856f5c5a4
I20260811 19:16:49.108619 3257 tablet_service.cc:1514] Processing DeleteTablet for tablet 356a2bde46834db7acfa5c2856f5c5a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:16:49 UTC) from {username='slave'} at 127.0.0.1:53480
I20260811 19:16:49.108501 3034 tablet_service.cc:1514] Processing DeleteTablet for tablet 356a2bde46834db7acfa5c2856f5c5a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:16:49 UTC) from {username='slave'} at 127.0.0.1:42574
I20260811 19:16:49.109903 3457 tablet_replica.cc:330] T 356a2bde46834db7acfa5c2856f5c5a4 P a42f5d12412943c1af9e1f31f267ed6b: stopping tablet replica
I20260811 19:16:49.110236 2811 tablet_service.cc:1514] Processing DeleteTablet for tablet 356a2bde46834db7acfa5c2856f5c5a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:16:49 UTC) from {username='slave'} at 127.0.0.1:35450
I20260811 19:16:49.110409 3457 raft_consensus.cc:2229] T 356a2bde46834db7acfa5c2856f5c5a4 P a42f5d12412943c1af9e1f31f267ed6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:16:49.110976 3457 raft_consensus.cc:2258] T 356a2bde46834db7acfa5c2856f5c5a4 P a42f5d12412943c1af9e1f31f267ed6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:16:49.113310 3457 ts_tablet_manager.cc:1905] T 356a2bde46834db7acfa5c2856f5c5a4 P a42f5d12412943c1af9e1f31f267ed6b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:16:49.115203 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 2699
I20260811 19:16:49.120878 3457 ts_tablet_manager.cc:1918] T 356a2bde46834db7acfa5c2856f5c5a4 P a42f5d12412943c1af9e1f31f267ed6b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:16:49.121064 3457 log.cc:1196] T 356a2bde46834db7acfa5c2856f5c5a4 P a42f5d12412943c1af9e1f31f267ed6b: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/ts-1/wal/wals/356a2bde46834db7acfa5c2856f5c5a4
I20260811 19:16:49.121526 3458 tablet_replica.cc:330] T 356a2bde46834db7acfa5c2856f5c5a4 P 090a3b7616af4f83b1b3f6f1d82eb380: stopping tablet replica
I20260811 19:16:49.121582 3457 ts_tablet_manager.cc:1939] T 356a2bde46834db7acfa5c2856f5c5a4 P a42f5d12412943c1af9e1f31f267ed6b: Deleting consensus metadata
I20260811 19:16:49.122189 3458 raft_consensus.cc:2229] T 356a2bde46834db7acfa5c2856f5c5a4 P 090a3b7616af4f83b1b3f6f1d82eb380 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:16:49.122857 3458 raft_consensus.cc:2258] T 356a2bde46834db7acfa5c2856f5c5a4 P 090a3b7616af4f83b1b3f6f1d82eb380 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:16:49.123757 2570 catalog_manager.cc:4841] TS a42f5d12412943c1af9e1f31f267ed6b (127.3.61.130:43821): tablet 356a2bde46834db7acfa5c2856f5c5a4 (table test-table [id=e97ac63142364dd084cf8abfffc9b2ba]) successfully deleted
I20260811 19:16:49.125422 3458 ts_tablet_manager.cc:1905] T 356a2bde46834db7acfa5c2856f5c5a4 P 090a3b7616af4f83b1b3f6f1d82eb380: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:16:49.130465 3456 tablet_replica.cc:330] T 356a2bde46834db7acfa5c2856f5c5a4 P 9af54edecfbf49b1be008d796de5a187: stopping tablet replica
I20260811 19:16:49.130929 3456 raft_consensus.cc:2229] T 356a2bde46834db7acfa5c2856f5c5a4 P 9af54edecfbf49b1be008d796de5a187 [term 1 LEADER]: Raft consensus shutting down.
I20260811 19:16:49.131699 3456 raft_consensus.cc:2258] T 356a2bde46834db7acfa5c2856f5c5a4 P 9af54edecfbf49b1be008d796de5a187 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:16:49.133803 3456 ts_tablet_manager.cc:1905] T 356a2bde46834db7acfa5c2856f5c5a4 P 9af54edecfbf49b1be008d796de5a187: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:16:49.133795 3458 ts_tablet_manager.cc:1918] T 356a2bde46834db7acfa5c2856f5c5a4 P 090a3b7616af4f83b1b3f6f1d82eb380: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:16:49.133985 3458 log.cc:1196] T 356a2bde46834db7acfa5c2856f5c5a4 P 090a3b7616af4f83b1b3f6f1d82eb380: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/ts-0/wal/wals/356a2bde46834db7acfa5c2856f5c5a4
I20260811 19:16:49.134605 3458 ts_tablet_manager.cc:1939] T 356a2bde46834db7acfa5c2856f5c5a4 P 090a3b7616af4f83b1b3f6f1d82eb380: Deleting consensus metadata
I20260811 19:16:49.136715 2569 catalog_manager.cc:4841] TS 090a3b7616af4f83b1b3f6f1d82eb380 (127.3.61.129:33999): tablet 356a2bde46834db7acfa5c2856f5c5a4 (table test-table [id=e97ac63142364dd084cf8abfffc9b2ba]) successfully deleted
I20260811 19:16:49.141067 3456 ts_tablet_manager.cc:1918] T 356a2bde46834db7acfa5c2856f5c5a4 P 9af54edecfbf49b1be008d796de5a187: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:16:49.141244 3456 log.cc:1196] T 356a2bde46834db7acfa5c2856f5c5a4 P 9af54edecfbf49b1be008d796de5a187: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786475454804547-3318-0/minicluster-data/ts-2/wal/wals/356a2bde46834db7acfa5c2856f5c5a4
I20260811 19:16:49.141795 3456 ts_tablet_manager.cc:1939] T 356a2bde46834db7acfa5c2856f5c5a4 P 9af54edecfbf49b1be008d796de5a187: Deleting consensus metadata
I20260811 19:16:49.143586 3412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:49.116900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35456 (local address 127.3.61.129:33999)
0811 19:16:49.117140 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:49.117148 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:49.134609 (+ 17461us) server_negotiation.cc:407] Connection header received
0811 19:16:49.135003 (+ 394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:49.135009 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:49.135108 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:49.135185 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:16:49.135911 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:49.136953 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:49.138655 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:49.139229 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:49.139358 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:49.140845 (+ 1487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:16:49.140857 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:16:49.140878 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:16:49.142067 (+ 1189us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:16:49.142189 (+ 122us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:16:49.142250 (+ 61us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:16:49.142402 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:49.142917 (+ 515us) server_negotiation.cc:299] Negotiation successful
0811 19:16:49.143156 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 19:16:49.143831 2568 catalog_manager.cc:4841] TS 9af54edecfbf49b1be008d796de5a187 (127.3.61.131:45191): tablet 356a2bde46834db7acfa5c2856f5c5a4 (table test-table [id=e97ac63142364dd084cf8abfffc9b2ba]) successfully deleted
I20260811 19:16:49.145556 2917 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:16:49.244709 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 2699
I20260811 19:16:49.272679 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 2930
I20260811 19:16:49.281288 3418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:49.273559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42584 (local address 127.3.61.130:43821)
0811 19:16:49.273779 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:49.273786 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:49.273806 (+ 20us) server_negotiation.cc:407] Connection header received
0811 19:16:49.273872 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:49.273878 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:49.273969 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:49.274055 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:16:49.274500 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:49.275529 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:49.276987 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:49.277538 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:49.277717 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:49.279389 (+ 1672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:16:49.279399 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:16:49.279413 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:16:49.280138 (+ 725us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:16:49.280195 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:16:49.280220 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:16:49.280356 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:49.280742 (+ 386us) server_negotiation.cc:299] Negotiation successful
0811 19:16:49.280905 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 19:16:49.282843 3143 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:16:49.385504 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 2930
I20260811 19:16:49.425472 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 3161
I20260811 19:16:49.434913 3416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:49.426784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53502 (local address 127.3.61.131:45191)
0811 19:16:49.426996 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:49.427002 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:49.427035 (+ 33us) server_negotiation.cc:407] Connection header received
0811 19:16:49.427333 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:49.427340 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:49.427473 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:49.427634 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:16:49.428129 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:49.429381 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:49.430849 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:49.431535 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:49.431675 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:49.432993 (+ 1318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:16:49.433006 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:16:49.433026 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:16:49.434002 (+ 976us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:16:49.434074 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:16:49.434101 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:16:49.434232 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:49.434396 (+ 164us) server_negotiation.cc:299] Negotiation successful
0811 19:16:49.434524 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 19:16:49.435871 3378 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:16:49.566560 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 3161
I20260811 19:16:49.594826 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 2534
I20260811 19:16:49.595932 2640 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:16:49.716894 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 2534
2026-08-11T19:16:49Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (3231 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:16:49 dist-test-slave-t7g5 krb5kdc[3475](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:16:49 dist-test-slave-t7g5 krb5kdc[3475](info): set up 2 sockets
Aug 11 19:16:49 dist-test-slave-t7g5 krb5kdc[3475](info): commencing operation
krb5kdc: starting...
W20260811 19:16:51.057715 3318 mini_kdc.cc:121] Time spent starting KDC: real 1.286s user 0.005s sys 0.056s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Aug 11 19:16:51 dist-test-slave-t7g5 krb5kdc[3475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475811, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:16:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:16:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.61.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab.
Entry for principal kudu/127.3.61.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab.
I20260811 19:16:51.445909 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:35217
--webserver_interface=127.3.61.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:33233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.61.190:35217
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:16:51.840796 3490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:16:51.841121 3490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:16:51.841239 3490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:16:51.849905 3490 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:16:51.850047 3490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:16:51.850108 3490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:16:51.850195 3490 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:16:51.850252 3490 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:16:51.863178 3490 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:33233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.61.190:35217
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:35217
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.61.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:16:51.865139 3490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:16:51.867040 3490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:16:51.880839 3496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:16:51.881107 3498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:16:51.880882 3504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:16:51.882324 3490 server_base.cc:1034] running on GCE node
I20260811 19:16:51.883426 3490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:16:51.885641 3490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:16:51.886955 3490 hybrid_clock.cc:648] HybridClock initialized: now 1786475811886881 us; error 69 us; skew 500 ppm
I20260811 19:16:51.887648 3490 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:16:51 dist-test-slave-t7g5 krb5kdc[3475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475811, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:16:51.896868 3490 init.cc:377] Logged in from keytab as kudu/127.3.61.190@KRBTEST.COM (short username kudu)
I20260811 19:16:51.899947 3490 webserver.cc:460] Webserver started at http://127.3.61.190:42017/ using document root <none> and password file <none>
I20260811 19:16:51.901157 3490 fs_manager.cc:359] Metadata directory not provided
I20260811 19:16:51.901495 3490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:16:51.901917 3490 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:16:51.905272 3490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/master-0/data/instance:
uuid: "c7412a12ee314eed8d7ab0f1ce479530"
format_stamp: "Formatted at 2026-08-11 19:16:51 on dist-test-slave-t7g5"
server_key: "54204c18261cc097b94676de899295d4"
server_key_iv: "db66faa7d4ddb304a1c8808bcdae0798"
server_key_version: "encryptionkey@0"
I20260811 19:16:51.906145 3490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/master-0/wal/instance:
uuid: "c7412a12ee314eed8d7ab0f1ce479530"
format_stamp: "Formatted at 2026-08-11 19:16:51 on dist-test-slave-t7g5"
server_key: "54204c18261cc097b94676de899295d4"
server_key_iv: "db66faa7d4ddb304a1c8808bcdae0798"
server_key_version: "encryptionkey@0"
I20260811 19:16:51.912549 3490 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 19:16:51.916512 3517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:51.918215 3490 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:16:51.918406 3490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/master-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/master-0/wal
uuid: "c7412a12ee314eed8d7ab0f1ce479530"
format_stamp: "Formatted at 2026-08-11 19:16:51 on dist-test-slave-t7g5"
server_key: "54204c18261cc097b94676de899295d4"
server_key_iv: "db66faa7d4ddb304a1c8808bcdae0798"
server_key_version: "encryptionkey@0"
I20260811 19:16:51.918641 3490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:16:51.940311 3490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:16:51.946257 3490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:16:51.946578 3490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:16:51.967442 3490 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.190:35217
I20260811 19:16:51.967608 3609 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.190:35217 every 8 connection(s)
I20260811 19:16:51.968937 3490 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
I20260811 19:16:51.972838 3610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:16:51.976524 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 3490
I20260811 19:16:51.976699 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/master-0/wal/instance
I20260811 19:16:51.977144 3318 external_mini_cluster.cc:1428] Setting key 7e0a66320c36eabd936c5cf4a3b8bffe
I20260811 19:16:51.988493 3610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7412a12ee314eed8d7ab0f1ce479530: Bootstrap starting.
I20260811 19:16:51.992610 3610 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7412a12ee314eed8d7ab0f1ce479530: Neither blocks nor log segments found. Creating new log.
I20260811 19:16:51.994109 3610 log.cc:824] T 00000000000000000000000000000000 P c7412a12ee314eed8d7ab0f1ce479530: Log is configured to *not* fsync() on all Append() calls
Aug 11 19:16:51 dist-test-slave-t7g5 krb5kdc[3475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475811, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:16:51.997536 3610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7412a12ee314eed8d7ab0f1ce479530: No bootstrap required, opened a new log
I20260811 19:16:52.008246 3614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:51.980817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41270 (local address 127.3.61.190:35217)
0811 19:16:51.982099 (+ 1282us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:51.982115 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:51.982166 (+ 51us) server_negotiation.cc:407] Connection header received
0811 19:16:51.983905 (+ 1739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:51.983931 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:51.984405 (+ 474us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:51.984794 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:16:51.986819 (+ 2025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:51.988661 (+ 1842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:51.990779 (+ 2118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:51.991552 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:51.991716 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:51.997102 (+ 5386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:16:51.997112 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:16:51.997129 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:16:51.997170 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:16:52.002119 (+ 4949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:16:52.003182 (+ 1063us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:16:52.003191 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:16:52.003201 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:16:52.003452 (+ 251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:16:52.003919 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:16:52.003923 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:16:52.003925 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:16:52.004726 (+ 801us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:16:52.005400 (+ 674us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:52.006129 (+ 729us) server_negotiation.cc:299] Negotiation successful
0811 19:16:52.006694 (+ 565us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1025,"thread_start_us":667,"threads_started":1}
I20260811 19:16:52.008412 3610 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7412a12ee314eed8d7ab0f1ce479530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7412a12ee314eed8d7ab0f1ce479530" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35217 } }
I20260811 19:16:52.009053 3610 raft_consensus.cc:374] T 00000000000000000000000000000000 P c7412a12ee314eed8d7ab0f1ce479530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:16:52.009172 3610 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7412a12ee314eed8d7ab0f1ce479530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7412a12ee314eed8d7ab0f1ce479530, State: Initialized, Role: FOLLOWER
I20260811 19:16:52.010038 3610 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7412a12ee314eed8d7ab0f1ce479530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7412a12ee314eed8d7ab0f1ce479530" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35217 } }
I20260811 19:16:52.010330 3610 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7412a12ee314eed8d7ab0f1ce479530 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:16:52.010484 3610 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7412a12ee314eed8d7ab0f1ce479530 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:16:52.010669 3610 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7412a12ee314eed8d7ab0f1ce479530 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:16:52.013432 3610 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7412a12ee314eed8d7ab0f1ce479530 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7412a12ee314eed8d7ab0f1ce479530" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35217 } }
I20260811 19:16:52.014110 3610 leader_election.cc:302] T 00000000000000000000000000000000 P c7412a12ee314eed8d7ab0f1ce479530 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7412a12ee314eed8d7ab0f1ce479530; no voters:
I20260811 19:16:52.014786 3610 leader_election.cc:288] T 00000000000000000000000000000000 P c7412a12ee314eed8d7ab0f1ce479530 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:16:52.014925 3617 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7412a12ee314eed8d7ab0f1ce479530 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:16:52.015743 3617 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7412a12ee314eed8d7ab0f1ce479530 [term 1 LEADER]: Becoming Leader. State: Replica: c7412a12ee314eed8d7ab0f1ce479530, State: Running, Role: LEADER
I20260811 19:16:52.016403 3617 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7412a12ee314eed8d7ab0f1ce479530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7412a12ee314eed8d7ab0f1ce479530" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35217 } }
I20260811 19:16:52.017243 3610 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7412a12ee314eed8d7ab0f1ce479530 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:16:52.019826 3623 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7412a12ee314eed8d7ab0f1ce479530 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7412a12ee314eed8d7ab0f1ce479530. Latest consensus state: current_term: 1 leader_uuid: "c7412a12ee314eed8d7ab0f1ce479530" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7412a12ee314eed8d7ab0f1ce479530" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35217 } } }
I20260811 19:16:52.020243 3623 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7412a12ee314eed8d7ab0f1ce479530 [sys.catalog]: This master's current role is: LEADER
I20260811 19:16:52.020835 3622 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7412a12ee314eed8d7ab0f1ce479530 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c7412a12ee314eed8d7ab0f1ce479530" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7412a12ee314eed8d7ab0f1ce479530" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35217 } } }
I20260811 19:16:52.021157 3622 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7412a12ee314eed8d7ab0f1ce479530 [sys.catalog]: This master's current role is: LEADER
I20260811 19:16:52.027164 3627 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:16:52.034348 3627 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:16:52.046397 3627 catalog_manager.cc:1347] Generated new cluster ID: 3ce1e11e3fd54db9a2803ac0dfcd34b8
I20260811 19:16:52.046543 3627 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:16:52.060752 3627 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:16:52.062556 3627 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:16:52.074168 3627 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c7412a12ee314eed8d7ab0f1ce479530: Generated new TSK 0
I20260811 19:16:52.075392 3627 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.61.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab.
Entry for principal kudu/127.3.61.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab.
I20260811 19:16:52.165839 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.129:0
--local_ip_for_outbound_sockets=127.3.61.129
--webserver_interface=127.3.61.129
--webserver_port=0
--tserver_master_addrs=127.3.61.190:35217
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab
--principal=kudu/127.3.61.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:33233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:16:52.570762 3656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:16:52.571141 3656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:16:52.571280 3656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:16:52.579602 3656 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:16:52.579779 3656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:16:52.579950 3656 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.129
I20260811 19:16:52.591513 3656 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:33233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.129.keytab
--principal=kudu/127.3.61.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.61.129
--webserver_port=0
--tserver_master_addrs=127.3.61.190:35217
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.129
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:16:52.593492 3656 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:16:52.595110 3656 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:16:52.610078 3665 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:16:52.610139 3668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:16:52.610148 3672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:16:52.611034 3656 server_base.cc:1034] running on GCE node
I20260811 19:16:52.611735 3656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:16:52.612836 3656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:16:52.614226 3656 hybrid_clock.cc:648] HybridClock initialized: now 1786475812614080 us; error 124 us; skew 500 ppm
I20260811 19:16:52.614601 3656 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:16:52 dist-test-slave-t7g5 krb5kdc[3475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475812, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:16:52.618418 3656 init.cc:377] Logged in from keytab as kudu/127.3.61.129@KRBTEST.COM (short username kudu)
I20260811 19:16:52.620141 3656 webserver.cc:460] Webserver started at http://127.3.61.129:42481/ using document root <none> and password file <none>
I20260811 19:16:52.621004 3656 fs_manager.cc:359] Metadata directory not provided
I20260811 19:16:52.621155 3656 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:16:52.621495 3656 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:16:52.624825 3656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/ts-0/data/instance:
uuid: "e2959af0df3940e6a10ec47fb0de53e2"
format_stamp: "Formatted at 2026-08-11 19:16:52 on dist-test-slave-t7g5"
server_key: "b0f6bc8917e62f735c5ba3cb01bf9b2c"
server_key_iv: "0b152e8740ae3fc256e11e4f2c3013f1"
server_key_version: "encryptionkey@0"
I20260811 19:16:52.625801 3656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/ts-0/wal/instance:
uuid: "e2959af0df3940e6a10ec47fb0de53e2"
format_stamp: "Formatted at 2026-08-11 19:16:52 on dist-test-slave-t7g5"
server_key: "b0f6bc8917e62f735c5ba3cb01bf9b2c"
server_key_iv: "0b152e8740ae3fc256e11e4f2c3013f1"
server_key_version: "encryptionkey@0"
I20260811 19:16:52.631547 3656 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 19:16:52.635573 3683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:52.637120 3656 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 19:16:52.637317 3656 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/ts-0/wal
uuid: "e2959af0df3940e6a10ec47fb0de53e2"
format_stamp: "Formatted at 2026-08-11 19:16:52 on dist-test-slave-t7g5"
server_key: "b0f6bc8917e62f735c5ba3cb01bf9b2c"
server_key_iv: "0b152e8740ae3fc256e11e4f2c3013f1"
server_key_version: "encryptionkey@0"
I20260811 19:16:52.637547 3656 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:16:52.662030 3656 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:16:52.667124 3656 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:16:52.667445 3656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:16:52.668534 3656 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:16:52.670147 3656 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:16:52.670234 3656 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:52.670372 3656 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:16:52.670426 3656 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:52.707629 3656 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.129:38803
I20260811 19:16:52.707705 3879 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.129:38803 every 8 connection(s)
I20260811 19:16:52.708971 3656 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
I20260811 19:16:52.710861 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 3656
I20260811 19:16:52.711068 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/ts-0/wal/instance
I20260811 19:16:52.711588 3318 external_mini_cluster.cc:1428] Setting key 9adc96a33dcc0559767189e12b95b106
Aug 11 19:16:52 dist-test-slave-t7g5 krb5kdc[3475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475812, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.129@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:16:52.733829 3891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:52.712221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:56523 (local address 127.3.61.190:35217)
0811 19:16:52.712949 (+ 728us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:52.712954 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:52.714037 (+ 1083us) server_negotiation.cc:407] Connection header received
0811 19:16:52.716660 (+ 2623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:52.716666 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:52.716759 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:52.716899 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:16:52.718815 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:52.719762 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:52.721586 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:52.722161 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:52.722307 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:52.726285 (+ 3978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:16:52.726295 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:16:52.726300 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:16:52.726366 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:16:52.730319 (+ 3953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:16:52.731249 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:16:52.731254 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:16:52.731256 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:16:52.731407 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:16:52.731867 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:16:52.731871 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:16:52.731874 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:16:52.732178 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:16:52.732321 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:52.733132 (+ 811us) server_negotiation.cc:299] Negotiation successful
0811 19:16:52.733405 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":510,"threads_started":1}
I20260811 19:16:52.734881 3895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:52.712748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35217 (local address 127.3.61.129:56523)
0811 19:16:52.713845 (+ 1097us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:52.713911 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 19:16:52.716289 (+ 2378us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:16:52.717330 (+ 1041us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:16:52.717340 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:16:52.717973 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:16:52.718638 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:52.718658 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:52.719965 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:52.719970 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:52.721424 (+ 1454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:52.721439 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:52.722475 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:52.722482 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:52.722728 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:52.723452 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:16:52.723483 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:16:52.725840 (+ 2357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:16:52.730536 (+ 4696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:16:52.730547 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:16:52.730564 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:16:52.731104 (+ 540us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:16:52.731522 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:16:52.731526 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:16:52.731528 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:16:52.731741 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:16:52.732314 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:16:52.732323 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:16:52.732708 (+ 385us) client_negotiation.cc:769] Sending connection context
0811 19:16:52.732982 (+ 274us) client_negotiation.cc:240] Negotiation successful
0811 19:16:52.733578 (+ 596us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":776,"thread_start_us":365,"threads_started":1}
I20260811 19:16:52.736804 3880 heartbeater.cc:347] Connected to a master server at 127.3.61.190:35217
I20260811 19:16:52.737355 3880 heartbeater.cc:464] Registering TS with master...
I20260811 19:16:52.738592 3880 heartbeater.cc:511] Master 127.3.61.190:35217 requested a full tablet report, sending...
I20260811 19:16:52.741585 3555 ts_manager.cc:194] Registered new tserver with Master: e2959af0df3940e6a10ec47fb0de53e2 (127.3.61.129:38803)
I20260811 19:16:52.743239 3555 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.129@KRBTEST.COM'} at 127.3.61.129:56523
WARNING: no policy specified for kudu/127.3.61.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab.
Entry for principal kudu/127.3.61.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab.
I20260811 19:16:52.794792 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.130:0
--local_ip_for_outbound_sockets=127.3.61.130
--webserver_interface=127.3.61.130
--webserver_port=0
--tserver_master_addrs=127.3.61.190:35217
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab
--principal=kudu/127.3.61.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:33233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:16:53.181061 3900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:16:53.181732 3900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:16:53.181962 3900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:16:53.190244 3900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:16:53.190382 3900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:16:53.190534 3900 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.130
I20260811 19:16:53.201297 3900 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:33233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.130.keytab
--principal=kudu/127.3.61.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.61.130
--webserver_port=0
--tserver_master_addrs=127.3.61.190:35217
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.130
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:16:53.202804 3900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:16:53.204201 3900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:16:53.218748 3908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:16:53.218796 3913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:16:53.218658 3909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:16:53.220218 3900 server_base.cc:1034] running on GCE node
I20260811 19:16:53.221072 3900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:16:53.222200 3900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:16:53.223444 3900 hybrid_clock.cc:648] HybridClock initialized: now 1786475813223376 us; error 51 us; skew 500 ppm
I20260811 19:16:53.223810 3900 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:16:53 dist-test-slave-t7g5 krb5kdc[3475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475813, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:16:53.227459 3900 init.cc:377] Logged in from keytab as kudu/127.3.61.130@KRBTEST.COM (short username kudu)
I20260811 19:16:53.229331 3900 webserver.cc:460] Webserver started at http://127.3.61.130:37697/ using document root <none> and password file <none>
I20260811 19:16:53.230183 3900 fs_manager.cc:359] Metadata directory not provided
I20260811 19:16:53.230326 3900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:16:53.230664 3900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:16:53.233922 3900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/ts-1/data/instance:
uuid: "9c969a1b68b7450c8391b2e525199673"
format_stamp: "Formatted at 2026-08-11 19:16:53 on dist-test-slave-t7g5"
server_key: "e426515a3fd312f2d46c3fb507a4fd26"
server_key_iv: "26eea88c88033f9f546003c6440317ce"
server_key_version: "encryptionkey@0"
I20260811 19:16:53.234814 3900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/ts-1/wal/instance:
uuid: "9c969a1b68b7450c8391b2e525199673"
format_stamp: "Formatted at 2026-08-11 19:16:53 on dist-test-slave-t7g5"
server_key: "e426515a3fd312f2d46c3fb507a4fd26"
server_key_iv: "26eea88c88033f9f546003c6440317ce"
server_key_version: "encryptionkey@0"
I20260811 19:16:53.241524 3900 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260811 19:16:53.247212 3924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:53.249539 3900 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.001s
I20260811 19:16:53.249883 3900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/ts-1/wal
uuid: "9c969a1b68b7450c8391b2e525199673"
format_stamp: "Formatted at 2026-08-11 19:16:53 on dist-test-slave-t7g5"
server_key: "e426515a3fd312f2d46c3fb507a4fd26"
server_key_iv: "26eea88c88033f9f546003c6440317ce"
server_key_version: "encryptionkey@0"
I20260811 19:16:53.250167 3900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:16:53.263314 3900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:16:53.269711 3900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:16:53.270041 3900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:16:53.271129 3900 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:16:53.272706 3900 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:16:53.272799 3900 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:53.272902 3900 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:16:53.272965 3900 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:53.310153 3900 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.130:41819
I20260811 19:16:53.310230 4127 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.130:41819 every 8 connection(s)
I20260811 19:16:53.311553 3900 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
I20260811 19:16:53.317711 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 3900
I20260811 19:16:53.317940 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/ts-1/wal/instance
I20260811 19:16:53.318509 3318 external_mini_cluster.cc:1428] Setting key ce0c7b7015f938d8fe46159f2d8ed70c
Aug 11 19:16:53 dist-test-slave-t7g5 krb5kdc[3475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475813, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.130@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:16:53.334308 4133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:53.314349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:56497 (local address 127.3.61.190:35217)
0811 19:16:53.314772 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:53.314777 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:53.315690 (+ 913us) server_negotiation.cc:407] Connection header received
0811 19:16:53.317814 (+ 2124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:53.317822 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:53.317934 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:53.318067 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:16:53.320173 (+ 2106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:53.320995 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:53.322752 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:53.323191 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:53.323304 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:53.327451 (+ 4147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:16:53.327457 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:16:53.327461 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:16:53.327499 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:16:53.330990 (+ 3491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:16:53.331853 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:16:53.331857 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:16:53.331860 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:16:53.331987 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:16:53.332442 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:16:53.332446 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:16:53.332448 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:16:53.332755 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:16:53.332891 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:53.333684 (+ 793us) server_negotiation.cc:299] Negotiation successful
0811 19:16:53.333946 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":215,"threads_started":1}
I20260811 19:16:53.335330 4134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:53.314744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35217 (local address 127.3.61.130:56497)
0811 19:16:53.315512 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:53.315560 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 19:16:53.317438 (+ 1878us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:16:53.318469 (+ 1031us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:16:53.318479 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:16:53.319191 (+ 712us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:16:53.319973 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:53.319994 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:53.321172 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:53.321175 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:53.322600 (+ 1425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:53.322618 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:53.323440 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:53.323448 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:53.323692 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:53.324412 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:16:53.324438 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:16:53.327064 (+ 2626us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:16:53.331177 (+ 4113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:16:53.331188 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:16:53.331209 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:16:53.331712 (+ 503us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:16:53.332097 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:16:53.332102 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:16:53.332106 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:16:53.332318 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:16:53.332892 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:16:53.332901 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:16:53.333270 (+ 369us) client_negotiation.cc:769] Sending connection context
0811 19:16:53.333523 (+ 253us) client_negotiation.cc:240] Negotiation successful
0811 19:16:53.334130 (+ 607us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":292,"threads_started":1}
I20260811 19:16:53.336920 4129 heartbeater.cc:347] Connected to a master server at 127.3.61.190:35217
I20260811 19:16:53.337306 4129 heartbeater.cc:464] Registering TS with master...
I20260811 19:16:53.338186 4129 heartbeater.cc:511] Master 127.3.61.190:35217 requested a full tablet report, sending...
I20260811 19:16:53.340574 3555 ts_manager.cc:194] Registered new tserver with Master: 9c969a1b68b7450c8391b2e525199673 (127.3.61.130:41819)
I20260811 19:16:53.342015 3555 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.130@KRBTEST.COM'} at 127.3.61.130:56497
WARNING: no policy specified for kudu/127.3.61.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab.
Entry for principal kudu/127.3.61.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab.
I20260811 19:16:53.399093 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.131:0
--local_ip_for_outbound_sockets=127.3.61.131
--webserver_interface=127.3.61.131
--webserver_port=0
--tserver_master_addrs=127.3.61.190:35217
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab
--principal=kudu/127.3.61.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:33233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:16:53.747643 3880 heartbeater.cc:503] Master 127.3.61.190:35217 was elected leader, sending a full tablet report...
W20260811 19:16:53.772645 4137 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:16:53.772950 4137 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:16:53.773077 4137 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:16:53.780503 4137 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:16:53.780643 4137 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:16:53.780787 4137 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.131
I20260811 19:16:53.790963 4137 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:33233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.131.keytab
--principal=kudu/127.3.61.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.61.131
--webserver_port=0
--tserver_master_addrs=127.3.61.190:35217
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.131
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:16:53.792325 4137 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:16:53.793737 4137 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:16:53.807152 4151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:16:53.807206 4148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:16:53.807467 4147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:16:53.808619 4137 server_base.cc:1034] running on GCE node
I20260811 19:16:53.809576 4137 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:16:53.810922 4137 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:16:53.812294 4137 hybrid_clock.cc:648] HybridClock initialized: now 1786475813812220 us; error 49 us; skew 500 ppm
I20260811 19:16:53.812855 4137 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:16:53 dist-test-slave-t7g5 krb5kdc[3475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475813, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:16:53.816879 4137 init.cc:377] Logged in from keytab as kudu/127.3.61.131@KRBTEST.COM (short username kudu)
I20260811 19:16:53.818988 4137 webserver.cc:460] Webserver started at http://127.3.61.131:35085/ using document root <none> and password file <none>
I20260811 19:16:53.819793 4137 fs_manager.cc:359] Metadata directory not provided
I20260811 19:16:53.819932 4137 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:16:53.820258 4137 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:16:53.823522 4137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/ts-2/data/instance:
uuid: "07e27e88889e4d00b95c4b2c54c872f0"
format_stamp: "Formatted at 2026-08-11 19:16:53 on dist-test-slave-t7g5"
server_key: "2708bc75213ae6baecf5b6019b6b6bfc"
server_key_iv: "43613d3f488c5fbce1d13f1030ef57b6"
server_key_version: "encryptionkey@0"
I20260811 19:16:53.824361 4137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/ts-2/wal/instance:
uuid: "07e27e88889e4d00b95c4b2c54c872f0"
format_stamp: "Formatted at 2026-08-11 19:16:53 on dist-test-slave-t7g5"
server_key: "2708bc75213ae6baecf5b6019b6b6bfc"
server_key_iv: "43613d3f488c5fbce1d13f1030ef57b6"
server_key_version: "encryptionkey@0"
I20260811 19:16:53.830211 4137 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260811 19:16:53.834056 4167 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:53.835760 4137 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 19:16:53.835942 4137 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/ts-2/wal
uuid: "07e27e88889e4d00b95c4b2c54c872f0"
format_stamp: "Formatted at 2026-08-11 19:16:53 on dist-test-slave-t7g5"
server_key: "2708bc75213ae6baecf5b6019b6b6bfc"
server_key_iv: "43613d3f488c5fbce1d13f1030ef57b6"
server_key_version: "encryptionkey@0"
I20260811 19:16:53.836148 4137 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:16:53.857802 4137 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:16:53.864013 4137 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:16:53.864418 4137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:16:53.865980 4137 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:16:53.868153 4137 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:16:53.868273 4137 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:53.868427 4137 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:16:53.868485 4137 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:53.905038 4137 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.131:39947
I20260811 19:16:53.905102 4366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.131:39947 every 8 connection(s)
I20260811 19:16:53.906823 4137 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
I20260811 19:16:53.907980 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 4137
I20260811 19:16:53.908176 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/ts-2/wal/instance
I20260811 19:16:53.908715 3318 external_mini_cluster.cc:1428] Setting key 0d22965f0b10cc90c6df9c2bb14141d6
Aug 11 19:16:53 dist-test-slave-t7g5 krb5kdc[3475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475813, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.131@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:16:53.929858 4371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:53.909812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:58367 (local address 127.3.61.190:35217)
0811 19:16:53.910325 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:53.910331 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:53.911681 (+ 1350us) server_negotiation.cc:407] Connection header received
0811 19:16:53.913835 (+ 2154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:53.913839 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:53.913908 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:53.914013 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:16:53.915685 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:53.916520 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:53.918260 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:53.918695 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:53.918820 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:53.922615 (+ 3795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:16:53.922620 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:16:53.922624 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:16:53.922684 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:16:53.926273 (+ 3589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:16:53.927112 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:16:53.927116 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:16:53.927119 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:16:53.927255 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:16:53.927882 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:16:53.927896 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:16:53.927899 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:16:53.928217 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:16:53.928353 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:53.929269 (+ 916us) server_negotiation.cc:299] Negotiation successful
0811 19:16:53.929470 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":221,"threads_started":1}
I20260811 19:16:53.930721 4372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:53.910515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:35217 (local address 127.3.61.131:58367)
0811 19:16:53.911489 (+ 974us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:53.911558 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0811 19:16:53.913526 (+ 1968us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:16:53.914278 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:16:53.914286 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:16:53.914864 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:16:53.915486 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:53.915515 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:53.916681 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:53.916685 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:53.918092 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:53.918106 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:53.918968 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:53.918976 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:53.919187 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:53.919926 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:16:53.919956 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:16:53.922356 (+ 2400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:16:53.926445 (+ 4089us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:16:53.926454 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:16:53.926469 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:16:53.926966 (+ 497us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:16:53.927381 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:16:53.927385 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:16:53.927387 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:16:53.927719 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:16:53.928369 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:16:53.928376 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:16:53.928777 (+ 401us) client_negotiation.cc:769] Sending connection context
0811 19:16:53.929080 (+ 303us) client_negotiation.cc:240] Negotiation successful
0811 19:16:53.929537 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":643,"thread_start_us":399,"threads_started":1}
I20260811 19:16:53.932386 4367 heartbeater.cc:347] Connected to a master server at 127.3.61.190:35217
I20260811 19:16:53.932713 4367 heartbeater.cc:464] Registering TS with master...
I20260811 19:16:53.933503 4367 heartbeater.cc:511] Master 127.3.61.190:35217 requested a full tablet report, sending...
I20260811 19:16:53.935187 3555 ts_manager.cc:194] Registered new tserver with Master: 07e27e88889e4d00b95c4b2c54c872f0 (127.3.61.131:39947)
I20260811 19:16:53.936058 3555 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.61.131@KRBTEST.COM'} at 127.3.61.131:58367
I20260811 19:16:53.938601 3318 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 19:16:53.958468 4371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:53.944101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53842 (local address 127.3.61.190:35217)
0811 19:16:53.944336 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:53.944341 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:53.944770 (+ 429us) server_negotiation.cc:407] Connection header received
0811 19:16:53.944888 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:53.944893 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:53.944974 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:53.945057 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:16:53.946660 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:53.947686 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:53.949016 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:53.949474 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:53.949572 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:53.952412 (+ 2840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:16:53.952418 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:16:53.952421 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:16:53.952459 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:16:53.955809 (+ 3350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:16:53.956575 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:16:53.956579 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:16:53.956581 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:16:53.956733 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:16:53.957035 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:16:53.957038 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:16:53.957040 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:16:53.957419 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:16:53.957617 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:53.957956 (+ 339us) server_negotiation.cc:299] Negotiation successful
0811 19:16:53.958159 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260811 19:16:53.963943 3555 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53842:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 19:16:53.971308 3555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:16:53.997089 4393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:53.986181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34738 (local address 127.3.61.130:41819)
0811 19:16:53.988898 (+ 2717us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:53.988907 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:53.988933 (+ 26us) server_negotiation.cc:407] Connection header received
0811 19:16:53.989025 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:53.989032 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:53.989229 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:53.990148 (+ 919us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:16:53.990373 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:53.992008 (+ 1635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:53.994695 (+ 2687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:53.995880 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:53.996014 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:53.996217 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:53.996575 (+ 358us) server_negotiation.cc:299] Negotiation successful
0811 19:16:53.996748 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2552,"thread_start_us":2443,"threads_started":1}
I20260811 19:16:53.997129 4392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:53.985960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.130:41819 (local address 127.0.0.1:34738)
0811 19:16:53.986857 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:53.986899 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 19:16:53.987095 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:16:53.989422 (+ 2327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:16:53.989426 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:16:53.989444 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:16:53.989925 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:53.989940 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:53.992142 (+ 2202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:53.992147 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:53.994494 (+ 2347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:53.994510 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:53.996120 (+ 1610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:53.996127 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:53.996336 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:53.996391 (+ 55us) client_negotiation.cc:769] Sending connection context
0811 19:16:53.996526 (+ 135us) client_negotiation.cc:240] Negotiation successful
0811 19:16:53.996717 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"mutex_wait_us":22,"thread_start_us":973,"threads_started":1}
I20260811 19:16:53.997282 4396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:53.986749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44658 (local address 127.3.61.129:38803)
0811 19:16:53.987870 (+ 1121us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:53.987880 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:53.988518 (+ 638us) server_negotiation.cc:407] Connection header received
0811 19:16:53.988630 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:53.988638 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:53.988884 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:53.989067 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:16:53.990689 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:53.992160 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:53.994631 (+ 2471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:53.996104 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:53.996234 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:53.996450 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:53.996774 (+ 324us) server_negotiation.cc:299] Negotiation successful
0811 19:16:53.996950 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":942,"thread_start_us":1995,"threads_started":1}
I20260811 19:16:53.998106 4395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:53.986401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:38803 (local address 127.0.0.1:44658)
0811 19:16:53.988247 (+ 1846us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:53.988262 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 19:16:53.988398 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:16:53.989369 (+ 971us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:16:53.989378 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:16:53.989409 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:16:53.990513 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:53.990528 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:53.992284 (+ 1756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:53.992288 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:53.994428 (+ 2140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:53.994450 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:53.996378 (+ 1928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:53.996385 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:53.996556 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:53.996590 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 19:16:53.997605 (+ 1015us) client_negotiation.cc:240] Negotiation successful
0811 19:16:53.997768 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1741,"mutex_wait_us":114,"thread_start_us":1620,"threads_started":1}
I20260811 19:16:53.998471 4397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:53.987788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.131:39947 (local address 127.0.0.1:43530)
0811 19:16:53.990337 (+ 2549us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:53.990354 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 19:16:53.990484 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:16:53.991344 (+ 860us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:16:53.991348 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:16:53.991372 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:16:53.991737 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:53.991750 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:53.993709 (+ 1959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:53.993718 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:53.995852 (+ 2134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:53.995868 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:53.997724 (+ 1856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:53.997732 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:53.997895 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:53.997925 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 19:16:53.998025 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 19:16:53.998153 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2443,"thread_start_us":2261,"threads_started":1}
I20260811 19:16:53.998926 4400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:53.988485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43530 (local address 127.3.61.131:39947)
0811 19:16:53.990074 (+ 1589us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:53.990082 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:53.990454 (+ 372us) server_negotiation.cc:407] Connection header received
0811 19:16:53.990667 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:53.990674 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:53.990899 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:53.991075 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:16:53.991929 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:53.993522 (+ 1593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:53.996021 (+ 2499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:53.997346 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:53.997487 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:53.997725 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:53.998079 (+ 354us) server_negotiation.cc:299] Negotiation successful
0811 19:16:53.998248 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1416,"thread_start_us":1235,"threads_started":1}
I20260811 19:16:54.001515 4008 tablet_service.cc:1467] Processing CreateTablet for tablet f91e8683183d4ac5b3630d47a8dd0f1f (DEFAULT_TABLE table=test-table [id=37de6239c0b64f798f2c93c4d3162980]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:16:54.001825 3775 tablet_service.cc:1467] Processing CreateTablet for tablet f91e8683183d4ac5b3630d47a8dd0f1f (DEFAULT_TABLE table=test-table [id=37de6239c0b64f798f2c93c4d3162980]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:16:54.002446 4263 tablet_service.cc:1467] Processing CreateTablet for tablet f91e8683183d4ac5b3630d47a8dd0f1f (DEFAULT_TABLE table=test-table [id=37de6239c0b64f798f2c93c4d3162980]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:16:54.003126 4008 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f91e8683183d4ac5b3630d47a8dd0f1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:16:54.003701 3775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f91e8683183d4ac5b3630d47a8dd0f1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:16:54.003854 4263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f91e8683183d4ac5b3630d47a8dd0f1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:16:54.016590 4401 tablet_bootstrap.cc:492] T f91e8683183d4ac5b3630d47a8dd0f1f P e2959af0df3940e6a10ec47fb0de53e2: Bootstrap starting.
I20260811 19:16:54.020795 4401 tablet_bootstrap.cc:654] T f91e8683183d4ac5b3630d47a8dd0f1f P e2959af0df3940e6a10ec47fb0de53e2: Neither blocks nor log segments found. Creating new log.
I20260811 19:16:54.022435 4401 log.cc:824] T f91e8683183d4ac5b3630d47a8dd0f1f P e2959af0df3940e6a10ec47fb0de53e2: Log is configured to *not* fsync() on all Append() calls
I20260811 19:16:54.023959 4402 tablet_bootstrap.cc:492] T f91e8683183d4ac5b3630d47a8dd0f1f P 07e27e88889e4d00b95c4b2c54c872f0: Bootstrap starting.
I20260811 19:16:54.025348 4403 tablet_bootstrap.cc:492] T f91e8683183d4ac5b3630d47a8dd0f1f P 9c969a1b68b7450c8391b2e525199673: Bootstrap starting.
I20260811 19:16:54.026892 4401 tablet_bootstrap.cc:492] T f91e8683183d4ac5b3630d47a8dd0f1f P e2959af0df3940e6a10ec47fb0de53e2: No bootstrap required, opened a new log
I20260811 19:16:54.027166 4401 ts_tablet_manager.cc:1397] T f91e8683183d4ac5b3630d47a8dd0f1f P e2959af0df3940e6a10ec47fb0de53e2: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260811 19:16:54.027770 4402 tablet_bootstrap.cc:654] T f91e8683183d4ac5b3630d47a8dd0f1f P 07e27e88889e4d00b95c4b2c54c872f0: Neither blocks nor log segments found. Creating new log.
I20260811 19:16:54.028925 4403 tablet_bootstrap.cc:654] T f91e8683183d4ac5b3630d47a8dd0f1f P 9c969a1b68b7450c8391b2e525199673: Neither blocks nor log segments found. Creating new log.
I20260811 19:16:54.029243 4402 log.cc:824] T f91e8683183d4ac5b3630d47a8dd0f1f P 07e27e88889e4d00b95c4b2c54c872f0: Log is configured to *not* fsync() on all Append() calls
I20260811 19:16:54.030289 4403 log.cc:824] T f91e8683183d4ac5b3630d47a8dd0f1f P 9c969a1b68b7450c8391b2e525199673: Log is configured to *not* fsync() on all Append() calls
I20260811 19:16:54.033763 4402 tablet_bootstrap.cc:492] T f91e8683183d4ac5b3630d47a8dd0f1f P 07e27e88889e4d00b95c4b2c54c872f0: No bootstrap required, opened a new log
I20260811 19:16:54.033782 4403 tablet_bootstrap.cc:492] T f91e8683183d4ac5b3630d47a8dd0f1f P 9c969a1b68b7450c8391b2e525199673: No bootstrap required, opened a new log
I20260811 19:16:54.034072 4402 ts_tablet_manager.cc:1397] T f91e8683183d4ac5b3630d47a8dd0f1f P 07e27e88889e4d00b95c4b2c54c872f0: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.005s
I20260811 19:16:54.034072 4403 ts_tablet_manager.cc:1397] T f91e8683183d4ac5b3630d47a8dd0f1f P 9c969a1b68b7450c8391b2e525199673: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260811 19:16:54.035540 4401 raft_consensus.cc:348] T f91e8683183d4ac5b3630d47a8dd0f1f P e2959af0df3940e6a10ec47fb0de53e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c969a1b68b7450c8391b2e525199673" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41819 } } peers { permanent_uuid: "07e27e88889e4d00b95c4b2c54c872f0" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39947 } } peers { permanent_uuid: "e2959af0df3940e6a10ec47fb0de53e2" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 38803 } }
I20260811 19:16:54.035923 4401 raft_consensus.cc:374] T f91e8683183d4ac5b3630d47a8dd0f1f P e2959af0df3940e6a10ec47fb0de53e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:16:54.036031 4401 raft_consensus.cc:729] T f91e8683183d4ac5b3630d47a8dd0f1f P e2959af0df3940e6a10ec47fb0de53e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2959af0df3940e6a10ec47fb0de53e2, State: Initialized, Role: FOLLOWER
I20260811 19:16:54.036880 4401 consensus_queue.cc:260] T f91e8683183d4ac5b3630d47a8dd0f1f P e2959af0df3940e6a10ec47fb0de53e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c969a1b68b7450c8391b2e525199673" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41819 } } peers { permanent_uuid: "07e27e88889e4d00b95c4b2c54c872f0" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39947 } } peers { permanent_uuid: "e2959af0df3940e6a10ec47fb0de53e2" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 38803 } }
I20260811 19:16:54.038802 4401 ts_tablet_manager.cc:1428] T f91e8683183d4ac5b3630d47a8dd0f1f P e2959af0df3940e6a10ec47fb0de53e2: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260811 19:16:54.045485 4403 raft_consensus.cc:348] T f91e8683183d4ac5b3630d47a8dd0f1f P 9c969a1b68b7450c8391b2e525199673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c969a1b68b7450c8391b2e525199673" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41819 } } peers { permanent_uuid: "07e27e88889e4d00b95c4b2c54c872f0" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39947 } } peers { permanent_uuid: "e2959af0df3940e6a10ec47fb0de53e2" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 38803 } }
I20260811 19:16:54.045404 4402 raft_consensus.cc:348] T f91e8683183d4ac5b3630d47a8dd0f1f P 07e27e88889e4d00b95c4b2c54c872f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c969a1b68b7450c8391b2e525199673" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41819 } } peers { permanent_uuid: "07e27e88889e4d00b95c4b2c54c872f0" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39947 } } peers { permanent_uuid: "e2959af0df3940e6a10ec47fb0de53e2" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 38803 } }
I20260811 19:16:54.045997 4403 raft_consensus.cc:374] T f91e8683183d4ac5b3630d47a8dd0f1f P 9c969a1b68b7450c8391b2e525199673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:16:54.045997 4402 raft_consensus.cc:374] T f91e8683183d4ac5b3630d47a8dd0f1f P 07e27e88889e4d00b95c4b2c54c872f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:16:54.046113 4402 raft_consensus.cc:729] T f91e8683183d4ac5b3630d47a8dd0f1f P 07e27e88889e4d00b95c4b2c54c872f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07e27e88889e4d00b95c4b2c54c872f0, State: Initialized, Role: FOLLOWER
I20260811 19:16:54.046113 4403 raft_consensus.cc:729] T f91e8683183d4ac5b3630d47a8dd0f1f P 9c969a1b68b7450c8391b2e525199673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c969a1b68b7450c8391b2e525199673, State: Initialized, Role: FOLLOWER
I20260811 19:16:54.046801 4403 consensus_queue.cc:260] T f91e8683183d4ac5b3630d47a8dd0f1f P 9c969a1b68b7450c8391b2e525199673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c969a1b68b7450c8391b2e525199673" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41819 } } peers { permanent_uuid: "07e27e88889e4d00b95c4b2c54c872f0" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39947 } } peers { permanent_uuid: "e2959af0df3940e6a10ec47fb0de53e2" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 38803 } }
I20260811 19:16:54.046824 4402 consensus_queue.cc:260] T f91e8683183d4ac5b3630d47a8dd0f1f P 07e27e88889e4d00b95c4b2c54c872f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c969a1b68b7450c8391b2e525199673" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41819 } } peers { permanent_uuid: "07e27e88889e4d00b95c4b2c54c872f0" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39947 } } peers { permanent_uuid: "e2959af0df3940e6a10ec47fb0de53e2" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 38803 } }
I20260811 19:16:54.048097 4129 heartbeater.cc:503] Master 127.3.61.190:35217 was elected leader, sending a full tablet report...
I20260811 19:16:54.048874 4403 ts_tablet_manager.cc:1428] T f91e8683183d4ac5b3630d47a8dd0f1f P 9c969a1b68b7450c8391b2e525199673: Time spent starting tablet: real 0.015s user 0.013s sys 0.000s
I20260811 19:16:54.049027 4402 ts_tablet_manager.cc:1428] T f91e8683183d4ac5b3630d47a8dd0f1f P 07e27e88889e4d00b95c4b2c54c872f0: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260811 19:16:54.049376 4367 heartbeater.cc:503] Master 127.3.61.190:35217 was elected leader, sending a full tablet report...
W20260811 19:16:54.063774 4131 tablet.cc:2369] T f91e8683183d4ac5b3630d47a8dd0f1f P 9c969a1b68b7450c8391b2e525199673: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 19:16:54.158628 4368 tablet.cc:2369] T f91e8683183d4ac5b3630d47a8dd0f1f P 07e27e88889e4d00b95c4b2c54c872f0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 19:16:54.211833 3885 tablet.cc:2369] T f91e8683183d4ac5b3630d47a8dd0f1f P e2959af0df3940e6a10ec47fb0de53e2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:16:54.314157 4419 raft_consensus.cc:482] T f91e8683183d4ac5b3630d47a8dd0f1f P 9c969a1b68b7450c8391b2e525199673 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:16:54.314491 4419 raft_consensus.cc:504] T f91e8683183d4ac5b3630d47a8dd0f1f P 9c969a1b68b7450c8391b2e525199673 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c969a1b68b7450c8391b2e525199673" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41819 } } peers { permanent_uuid: "07e27e88889e4d00b95c4b2c54c872f0" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39947 } } peers { permanent_uuid: "e2959af0df3940e6a10ec47fb0de53e2" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 38803 } }
I20260811 19:16:54.316404 4419 leader_election.cc:288] T f91e8683183d4ac5b3630d47a8dd0f1f P 9c969a1b68b7450c8391b2e525199673 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07e27e88889e4d00b95c4b2c54c872f0 (127.3.61.131:39947), e2959af0df3940e6a10ec47fb0de53e2 (127.3.61.129:38803)
I20260811 19:16:54.325018 4396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:54.317349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:56127 (local address 127.3.61.129:38803)
0811 19:16:54.317568 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:54.317574 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:54.318459 (+ 885us) server_negotiation.cc:407] Connection header received
0811 19:16:54.318612 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:54.318617 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:54.318698 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:54.318795 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:16:54.319553 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:54.320672 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:54.323190 (+ 2518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:54.324085 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:54.324246 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:54.324326 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:54.324630 (+ 304us) server_negotiation.cc:299] Negotiation successful
0811 19:16:54.324764 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 19:16:54.325155 4422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:54.316854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:38803 (local address 127.3.61.130:56127)
0811 19:16:54.318361 (+ 1507us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:54.318375 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 19:16:54.318486 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:16:54.318835 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:16:54.318841 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:16:54.318867 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:16:54.319358 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:54.319371 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:54.320815 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:54.320820 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:54.322948 (+ 2128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:54.322966 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:54.324300 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:54.324306 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:54.324454 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:54.324479 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 19:16:54.324576 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 19:16:54.324724 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1386,"mutex_wait_us":6,"thread_start_us":1170,"threads_started":1}
I20260811 19:16:54.325384 4400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:54.317069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:34497 (local address 127.3.61.131:39947)
0811 19:16:54.317310 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:54.317317 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:54.317980 (+ 663us) server_negotiation.cc:407] Connection header received
0811 19:16:54.318395 (+ 415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:54.318402 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:54.318492 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:54.318590 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:16:54.319236 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:54.320302 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:54.323503 (+ 3201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:54.324331 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:54.324492 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:54.324569 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:54.324877 (+ 308us) server_negotiation.cc:299] Negotiation successful
0811 19:16:54.325013 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260811 19:16:54.325766 4421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:54.316733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.131:39947 (local address 127.3.61.130:34497)
0811 19:16:54.317858 (+ 1125us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:54.317879 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 19:16:54.318115 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:16:54.318622 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:16:54.318626 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:16:54.318649 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:16:54.319088 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:54.319103 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:54.320725 (+ 1622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:54.320735 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:54.323319 (+ 2584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:54.323333 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:54.324566 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:54.324572 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:54.324723 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:54.324746 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 19:16:54.324828 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 19:16:54.324962 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":986,"mutex_wait_us":26,"thread_start_us":767,"threads_started":1}
I20260811 19:16:54.326323 3809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f91e8683183d4ac5b3630d47a8dd0f1f" candidate_uuid: "9c969a1b68b7450c8391b2e525199673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2959af0df3940e6a10ec47fb0de53e2" is_pre_election: true
I20260811 19:16:54.326805 4299 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f91e8683183d4ac5b3630d47a8dd0f1f" candidate_uuid: "9c969a1b68b7450c8391b2e525199673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07e27e88889e4d00b95c4b2c54c872f0" is_pre_election: true
I20260811 19:16:54.326993 3809 raft_consensus.cc:2454] T f91e8683183d4ac5b3630d47a8dd0f1f P e2959af0df3940e6a10ec47fb0de53e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c969a1b68b7450c8391b2e525199673 in term 0.
I20260811 19:16:54.327054 4416 raft_consensus.cc:482] T f91e8683183d4ac5b3630d47a8dd0f1f P 07e27e88889e4d00b95c4b2c54c872f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:16:54.327325 4416 raft_consensus.cc:504] T f91e8683183d4ac5b3630d47a8dd0f1f P 07e27e88889e4d00b95c4b2c54c872f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c969a1b68b7450c8391b2e525199673" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41819 } } peers { permanent_uuid: "07e27e88889e4d00b95c4b2c54c872f0" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39947 } } peers { permanent_uuid: "e2959af0df3940e6a10ec47fb0de53e2" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 38803 } }
I20260811 19:16:54.327747 3926 leader_election.cc:302] T f91e8683183d4ac5b3630d47a8dd0f1f P 9c969a1b68b7450c8391b2e525199673 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c969a1b68b7450c8391b2e525199673, e2959af0df3940e6a10ec47fb0de53e2; no voters:
I20260811 19:16:54.328132 4419 raft_consensus.cc:2781] T f91e8683183d4ac5b3630d47a8dd0f1f P 9c969a1b68b7450c8391b2e525199673 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:16:54.328065 4299 raft_consensus.cc:2454] T f91e8683183d4ac5b3630d47a8dd0f1f P 07e27e88889e4d00b95c4b2c54c872f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c969a1b68b7450c8391b2e525199673 in term 0.
I20260811 19:16:54.328266 4419 raft_consensus.cc:482] T f91e8683183d4ac5b3630d47a8dd0f1f P 9c969a1b68b7450c8391b2e525199673 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:16:54.328369 4419 raft_consensus.cc:3037] T f91e8683183d4ac5b3630d47a8dd0f1f P 9c969a1b68b7450c8391b2e525199673 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:16:54.329164 4416 leader_election.cc:288] T f91e8683183d4ac5b3630d47a8dd0f1f P 07e27e88889e4d00b95c4b2c54c872f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c969a1b68b7450c8391b2e525199673 (127.3.61.130:41819), e2959af0df3940e6a10ec47fb0de53e2 (127.3.61.129:38803)
I20260811 19:16:54.331216 4419 raft_consensus.cc:504] T f91e8683183d4ac5b3630d47a8dd0f1f P 9c969a1b68b7450c8391b2e525199673 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c969a1b68b7450c8391b2e525199673" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41819 } } peers { permanent_uuid: "07e27e88889e4d00b95c4b2c54c872f0" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39947 } } peers { permanent_uuid: "e2959af0df3940e6a10ec47fb0de53e2" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 38803 } }
I20260811 19:16:54.331880 4419 leader_election.cc:288] T f91e8683183d4ac5b3630d47a8dd0f1f P 9c969a1b68b7450c8391b2e525199673 [CANDIDATE]: Term 1 election: Requested vote from peers 07e27e88889e4d00b95c4b2c54c872f0 (127.3.61.131:39947), e2959af0df3940e6a10ec47fb0de53e2 (127.3.61.129:38803)
I20260811 19:16:54.332793 3809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f91e8683183d4ac5b3630d47a8dd0f1f" candidate_uuid: "9c969a1b68b7450c8391b2e525199673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2959af0df3940e6a10ec47fb0de53e2"
I20260811 19:16:54.332800 4299 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f91e8683183d4ac5b3630d47a8dd0f1f" candidate_uuid: "9c969a1b68b7450c8391b2e525199673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07e27e88889e4d00b95c4b2c54c872f0"
I20260811 19:16:54.333061 4299 raft_consensus.cc:3037] T f91e8683183d4ac5b3630d47a8dd0f1f P 07e27e88889e4d00b95c4b2c54c872f0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:16:54.333033 3809 raft_consensus.cc:3037] T f91e8683183d4ac5b3630d47a8dd0f1f P e2959af0df3940e6a10ec47fb0de53e2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:16:54.336683 3809 raft_consensus.cc:2454] T f91e8683183d4ac5b3630d47a8dd0f1f P e2959af0df3940e6a10ec47fb0de53e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c969a1b68b7450c8391b2e525199673 in term 1.
I20260811 19:16:54.336792 4299 raft_consensus.cc:2454] T f91e8683183d4ac5b3630d47a8dd0f1f P 07e27e88889e4d00b95c4b2c54c872f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c969a1b68b7450c8391b2e525199673 in term 1.
I20260811 19:16:54.337353 3929 leader_election.cc:302] T f91e8683183d4ac5b3630d47a8dd0f1f P 9c969a1b68b7450c8391b2e525199673 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07e27e88889e4d00b95c4b2c54c872f0, 9c969a1b68b7450c8391b2e525199673; no voters:
I20260811 19:16:54.337481 4393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:54.329819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:34977 (local address 127.3.61.130:41819)
0811 19:16:54.330114 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:54.330120 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:54.330140 (+ 20us) server_negotiation.cc:407] Connection header received
0811 19:16:54.330204 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:54.330209 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:54.330286 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:54.330368 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:16:54.330830 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:54.332080 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:54.334564 (+ 2484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:54.335783 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:54.335930 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:54.336006 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:54.336888 (+ 882us) server_negotiation.cc:299] Negotiation successful
0811 19:16:54.337030 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147}
I20260811 19:16:54.337882 4419 raft_consensus.cc:2781] T f91e8683183d4ac5b3630d47a8dd0f1f P 9c969a1b68b7450c8391b2e525199673 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:16:54.338400 4419 raft_consensus.cc:686] T f91e8683183d4ac5b3630d47a8dd0f1f P 9c969a1b68b7450c8391b2e525199673 [term 1 LEADER]: Becoming Leader. State: Replica: 9c969a1b68b7450c8391b2e525199673, State: Running, Role: LEADER
I20260811 19:16:54.339020 4419 consensus_queue.cc:237] T f91e8683183d4ac5b3630d47a8dd0f1f P 9c969a1b68b7450c8391b2e525199673 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c969a1b68b7450c8391b2e525199673" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41819 } } peers { permanent_uuid: "07e27e88889e4d00b95c4b2c54c872f0" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39947 } } peers { permanent_uuid: "e2959af0df3940e6a10ec47fb0de53e2" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 38803 } }
I20260811 19:16:54.340430 4396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:54.330171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:47043 (local address 127.3.61.129:38803)
0811 19:16:54.330363 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:54.330370 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:54.332685 (+ 2315us) server_negotiation.cc:407] Connection header received
0811 19:16:54.332962 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:54.332967 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:54.333047 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:54.333112 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:16:54.334312 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:54.335484 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:54.337769 (+ 2285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:54.339006 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:54.339195 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:54.339271 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:54.339997 (+ 726us) server_negotiation.cc:299] Negotiation successful
0811 19:16:54.340144 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 19:16:54.340473 4424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:54.329584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:38803 (local address 127.3.61.131:47043)
0811 19:16:54.332569 (+ 2985us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:54.332597 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 19:16:54.332797 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:16:54.333314 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:16:54.333319 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:16:54.333348 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:16:54.334085 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:54.334099 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:54.335709 (+ 1610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:54.335714 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:54.337592 (+ 1878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:54.337610 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:54.339473 (+ 1863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:54.339481 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:54.339732 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:54.339772 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 19:16:54.339903 (+ 131us) client_negotiation.cc:240] Negotiation successful
0811 19:16:54.340089 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2848,"thread_start_us":2628,"threads_started":1}
I20260811 19:16:54.341881 4372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:54.329427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.130:41819 (local address 127.3.61.131:34977)
0811 19:16:54.329610 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:54.329630 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 19:16:54.329791 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:16:54.330422 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:16:54.330428 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:16:54.330454 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:16:54.330659 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:54.330672 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:54.332294 (+ 1622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:54.332300 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:54.334360 (+ 2060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:54.334375 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:54.336094 (+ 1719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:54.336100 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:54.336333 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:54.336378 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 19:16:54.336478 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 19:16:54.336636 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":1}
I20260811 19:16:54.341854 3809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f91e8683183d4ac5b3630d47a8dd0f1f" candidate_uuid: "07e27e88889e4d00b95c4b2c54c872f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2959af0df3940e6a10ec47fb0de53e2" is_pre_election: true
I20260811 19:16:54.342481 3809 raft_consensus.cc:2379] T f91e8683183d4ac5b3630d47a8dd0f1f P e2959af0df3940e6a10ec47fb0de53e2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 07e27e88889e4d00b95c4b2c54c872f0 in current term 1: Already voted for candidate 9c969a1b68b7450c8391b2e525199673 in this term.
I20260811 19:16:54.342912 4046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f91e8683183d4ac5b3630d47a8dd0f1f" candidate_uuid: "07e27e88889e4d00b95c4b2c54c872f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c969a1b68b7450c8391b2e525199673" is_pre_election: true
I20260811 19:16:54.344144 4178 leader_election.cc:302] T f91e8683183d4ac5b3630d47a8dd0f1f P 07e27e88889e4d00b95c4b2c54c872f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 07e27e88889e4d00b95c4b2c54c872f0; no voters: 9c969a1b68b7450c8391b2e525199673, e2959af0df3940e6a10ec47fb0de53e2
I20260811 19:16:54.344602 4416 raft_consensus.cc:2726] T f91e8683183d4ac5b3630d47a8dd0f1f P 07e27e88889e4d00b95c4b2c54c872f0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 19:16:54.344292 3555 catalog_manager.cc:5495] T f91e8683183d4ac5b3630d47a8dd0f1f P 9c969a1b68b7450c8391b2e525199673 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c969a1b68b7450c8391b2e525199673 (127.3.61.130). New cstate: current_term: 1 leader_uuid: "9c969a1b68b7450c8391b2e525199673" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c969a1b68b7450c8391b2e525199673" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 41819 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07e27e88889e4d00b95c4b2c54c872f0" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2959af0df3940e6a10ec47fb0de53e2" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 38803 } health_report { overall_health: UNKNOWN } } }
I20260811 19:16:54.385205 4393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:54.377403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34742 (local address 127.3.61.130:41819)
0811 19:16:54.377710 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:54.377716 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:54.377741 (+ 25us) server_negotiation.cc:407] Connection header received
0811 19:16:54.377859 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:54.377863 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:54.377987 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:54.378164 (+ 177us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:16:54.379082 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:54.380243 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:54.382017 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:54.382437 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:54.382548 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:54.383367 (+ 819us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:16:54.384156 (+ 789us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:16:54.384332 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:54.384677 (+ 345us) server_negotiation.cc:299] Negotiation successful
0811 19:16:54.384816 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260811 19:16:54.394975 3809 raft_consensus.cc:1261] T f91e8683183d4ac5b3630d47a8dd0f1f P e2959af0df3940e6a10ec47fb0de53e2 [term 1 FOLLOWER]: Refusing update from remote peer 9c969a1b68b7450c8391b2e525199673: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:16:54.395439 4299 raft_consensus.cc:1261] T f91e8683183d4ac5b3630d47a8dd0f1f P 07e27e88889e4d00b95c4b2c54c872f0 [term 1 FOLLOWER]: Refusing update from remote peer 9c969a1b68b7450c8391b2e525199673: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:16:54.396143 4425 consensus_queue.cc:1035] T f91e8683183d4ac5b3630d47a8dd0f1f P 9c969a1b68b7450c8391b2e525199673 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2959af0df3940e6a10ec47fb0de53e2" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 38803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:16:54.396669 4419 consensus_queue.cc:1035] T f91e8683183d4ac5b3630d47a8dd0f1f P 9c969a1b68b7450c8391b2e525199673 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07e27e88889e4d00b95c4b2c54c872f0" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:16:54.407146 4429 mvcc.cc:205] Tried to move back new op lower bound from 7317404935751471104 to 7317404935537602560. Current Snapshot: MvccSnapshot[applied={T|T < 7317404935751471104}]
I20260811 19:16:54.421834 3553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:16:54.422127 3553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:16:54.425884 3553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c7412a12ee314eed8d7ab0f1ce479530: Sending DeleteTablet for 3 replicas of tablet f91e8683183d4ac5b3630d47a8dd0f1f
I20260811 19:16:54.427276 4008 tablet_service.cc:1514] Processing DeleteTablet for tablet f91e8683183d4ac5b3630d47a8dd0f1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:16:54 UTC) from {username='kudu', principal='kudu/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:34738
I20260811 19:16:54.427312 3775 tablet_service.cc:1514] Processing DeleteTablet for tablet f91e8683183d4ac5b3630d47a8dd0f1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:16:54 UTC) from {username='kudu', principal='kudu/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:44658
I20260811 19:16:54.428005 4263 tablet_service.cc:1514] Processing DeleteTablet for tablet f91e8683183d4ac5b3630d47a8dd0f1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:16:54 UTC) from {username='kudu', principal='kudu/127.3.61.190@KRBTEST.COM'} at 127.0.0.1:43530
I20260811 19:16:54.428471 4457 tablet_replica.cc:330] T f91e8683183d4ac5b3630d47a8dd0f1f P e2959af0df3940e6a10ec47fb0de53e2: stopping tablet replica
I20260811 19:16:54.428871 4458 tablet_replica.cc:330] T f91e8683183d4ac5b3630d47a8dd0f1f P 07e27e88889e4d00b95c4b2c54c872f0: stopping tablet replica
I20260811 19:16:54.429293 4458 raft_consensus.cc:2229] T f91e8683183d4ac5b3630d47a8dd0f1f P 07e27e88889e4d00b95c4b2c54c872f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:16:54.429888 4458 raft_consensus.cc:2258] T f91e8683183d4ac5b3630d47a8dd0f1f P 07e27e88889e4d00b95c4b2c54c872f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:16:54.432191 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 3656
I20260811 19:16:54.432242 4458 ts_tablet_manager.cc:1905] T f91e8683183d4ac5b3630d47a8dd0f1f P 07e27e88889e4d00b95c4b2c54c872f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:16:54.439724 4457 raft_consensus.cc:2229] T f91e8683183d4ac5b3630d47a8dd0f1f P e2959af0df3940e6a10ec47fb0de53e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:16:54.439839 4458 ts_tablet_manager.cc:1918] T f91e8683183d4ac5b3630d47a8dd0f1f P 07e27e88889e4d00b95c4b2c54c872f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:16:54.440042 4458 log.cc:1196] T f91e8683183d4ac5b3630d47a8dd0f1f P 07e27e88889e4d00b95c4b2c54c872f0: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/ts-2/wal/wals/f91e8683183d4ac5b3630d47a8dd0f1f
I20260811 19:16:54.440845 4458 ts_tablet_manager.cc:1939] T f91e8683183d4ac5b3630d47a8dd0f1f P 07e27e88889e4d00b95c4b2c54c872f0: Deleting consensus metadata
I20260811 19:16:54.441128 4457 raft_consensus.cc:2258] T f91e8683183d4ac5b3630d47a8dd0f1f P e2959af0df3940e6a10ec47fb0de53e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:16:54.443013 3525 catalog_manager.cc:4841] TS 07e27e88889e4d00b95c4b2c54c872f0 (127.3.61.131:39947): tablet f91e8683183d4ac5b3630d47a8dd0f1f (table test-table [id=37de6239c0b64f798f2c93c4d3162980]) successfully deleted
I20260811 19:16:54.444433 4457 ts_tablet_manager.cc:1905] T f91e8683183d4ac5b3630d47a8dd0f1f P e2959af0df3940e6a10ec47fb0de53e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:16:54.445346 4453 tablet_replica.cc:330] T f91e8683183d4ac5b3630d47a8dd0f1f P 9c969a1b68b7450c8391b2e525199673: stopping tablet replica
I20260811 19:16:54.446237 4453 raft_consensus.cc:2229] T f91e8683183d4ac5b3630d47a8dd0f1f P 9c969a1b68b7450c8391b2e525199673 [term 1 LEADER]: Raft consensus shutting down.
I20260811 19:16:54.446967 4453 raft_consensus.cc:2258] T f91e8683183d4ac5b3630d47a8dd0f1f P 9c969a1b68b7450c8391b2e525199673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:16:54.449319 4453 ts_tablet_manager.cc:1905] T f91e8683183d4ac5b3630d47a8dd0f1f P 9c969a1b68b7450c8391b2e525199673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:16:54.451761 4457 ts_tablet_manager.cc:1918] T f91e8683183d4ac5b3630d47a8dd0f1f P e2959af0df3940e6a10ec47fb0de53e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:16:54.451946 4457 log.cc:1196] T f91e8683183d4ac5b3630d47a8dd0f1f P e2959af0df3940e6a10ec47fb0de53e2: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/ts-0/wal/wals/f91e8683183d4ac5b3630d47a8dd0f1f
I20260811 19:16:54.452527 4457 ts_tablet_manager.cc:1939] T f91e8683183d4ac5b3630d47a8dd0f1f P e2959af0df3940e6a10ec47fb0de53e2: Deleting consensus metadata
Aug 11 19:16:54 dist-test-slave-t7g5 krb5kdc[3475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475811, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.129@KRBTEST.COM
I20260811 19:16:54.454623 3520 catalog_manager.cc:4841] TS e2959af0df3940e6a10ec47fb0de53e2 (127.3.61.129:38803): tablet f91e8683183d4ac5b3630d47a8dd0f1f (table test-table [id=37de6239c0b64f798f2c93c4d3162980]) successfully deleted
I20260811 19:16:54.457353 4453 ts_tablet_manager.cc:1918] T f91e8683183d4ac5b3630d47a8dd0f1f P 9c969a1b68b7450c8391b2e525199673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:16:54.457533 4453 log.cc:1196] T f91e8683183d4ac5b3630d47a8dd0f1f P 9c969a1b68b7450c8391b2e525199673: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786475454804547-3318-0/minicluster-data/ts-1/wal/wals/f91e8683183d4ac5b3630d47a8dd0f1f
I20260811 19:16:54.458122 4453 ts_tablet_manager.cc:1939] T f91e8683183d4ac5b3630d47a8dd0f1f P 9c969a1b68b7450c8391b2e525199673: Deleting consensus metadata
I20260811 19:16:54.460254 3525 catalog_manager.cc:4841] TS 9c969a1b68b7450c8391b2e525199673 (127.3.61.130:41819): tablet f91e8683183d4ac5b3630d47a8dd0f1f (table test-table [id=37de6239c0b64f798f2c93c4d3162980]) successfully deleted
I20260811 19:16:54.463312 4396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:54.434017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44660 (local address 127.3.61.129:38803)
0811 19:16:54.434204 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:54.434211 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:54.440051 (+ 5840us) server_negotiation.cc:407] Connection header received
0811 19:16:54.441069 (+ 1018us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:54.441076 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:54.441194 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:54.441282 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:16:54.444326 (+ 3044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:54.445983 (+ 1657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:54.448059 (+ 2076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:54.448752 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:54.448886 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:54.455767 (+ 6881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:16:54.455782 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:16:54.455802 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:16:54.455857 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:16:54.459867 (+ 4010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:16:54.460813 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:16:54.460824 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:16:54.460828 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:16:54.461016 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:16:54.461485 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:16:54.461491 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:16:54.461495 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:16:54.462067 (+ 572us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:16:54.462397 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:54.462703 (+ 306us) server_negotiation.cc:299] Negotiation successful
0811 19:16:54.462955 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 19:16:54.464041 3871 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:16:54.557298 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 3656
I20260811 19:16:54.587051 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 3900
Aug 11 19:16:54 dist-test-slave-t7g5 krb5kdc[3475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475811, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.130@KRBTEST.COM
I20260811 19:16:54.607163 4393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:54.587847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34756 (local address 127.3.61.130:41819)
0811 19:16:54.588060 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:54.588066 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:54.588089 (+ 23us) server_negotiation.cc:407] Connection header received
0811 19:16:54.588240 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:54.588245 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:54.588341 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:54.588472 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:16:54.589927 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:54.591068 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:54.592569 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:54.593235 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:54.593377 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:54.598467 (+ 5090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:16:54.598490 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:16:54.598514 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:16:54.598570 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:16:54.603351 (+ 4781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:16:54.604321 (+ 970us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:16:54.604331 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:16:54.604335 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:16:54.604555 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:16:54.605262 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:16:54.605268 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:16:54.605272 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:16:54.605862 (+ 590us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:16:54.606175 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:54.606522 (+ 347us) server_negotiation.cc:299] Negotiation successful
0811 19:16:54.606785 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 19:16:54.608000 4120 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:16:54.730844 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 3900
I20260811 19:16:54.758450 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 4137
Aug 11 19:16:54 dist-test-slave-t7g5 krb5kdc[3475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475811, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.131@KRBTEST.COM
I20260811 19:16:54.777168 4400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:54.759266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43544 (local address 127.3.61.131:39947)
0811 19:16:54.759436 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:54.759441 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:54.759461 (+ 20us) server_negotiation.cc:407] Connection header received
0811 19:16:54.759524 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:54.759528 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:54.759595 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:54.759688 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:16:54.761725 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:54.762484 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:54.764125 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:54.764548 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:54.764650 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:54.769850 (+ 5200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:16:54.769871 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:16:54.769891 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:16:54.769939 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:16:54.773341 (+ 3402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:16:54.774335 (+ 994us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:16:54.774348 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:16:54.774351 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:16:54.774772 (+ 421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:16:54.775293 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:16:54.775297 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:16:54.775300 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:16:54.775869 (+ 569us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:16:54.776166 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:54.776634 (+ 468us) server_negotiation.cc:299] Negotiation successful
0811 19:16:54.776857 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 19:16:54.777882 4362 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:16:54.871625 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 4137
I20260811 19:16:54.897250 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 3490
I20260811 19:16:54.898805 3602 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:16:55.016505 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 3490
2026-08-11T19:16:55Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (5296 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-11T19:16:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:16:55Z Disabled control of system clock
I20260811 19:16:55.130656 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:44105
--webserver_interface=127.3.61.190
--webserver_port=0
--builtin_ntp_servers=127.3.61.148:39273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.61.190:44105
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 19:16:55.506083 4473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:16:55.506403 4473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:16:55.506487 4473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:16:55.515143 4473 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:16:55.515296 4473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:16:55.515337 4473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:16:55.515389 4473 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:16:55.515443 4473 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:16:55.525949 4473 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:39273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.61.190:44105
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:44105
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.61.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:16:55.527305 4473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:16:55.529189 4473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:16:55.542066 4480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:16:55.542196 4484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:16:55.542688 4479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:16:55.542793 4473 server_base.cc:1034] running on GCE node
I20260811 19:16:55.543717 4473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:16:55.545271 4473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:16:55.546510 4473 hybrid_clock.cc:648] HybridClock initialized: now 1786475815546465 us; error 39 us; skew 500 ppm
I20260811 19:16:55.546918 4473 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 19:16:55.549559 4473 webserver.cc:460] Webserver started at http://127.3.61.190:41591/ using document root <none> and password file <none>
I20260811 19:16:55.550325 4473 fs_manager.cc:359] Metadata directory not provided
I20260811 19:16:55.550438 4473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:16:55.550768 4473 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:16:55.554108 4473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/master-0/data/instance:
uuid: "41c487474a9b46629ca620a804fb5ff8"
format_stamp: "Formatted at 2026-08-11 19:16:55 on dist-test-slave-t7g5"
server_key: "b6f2ef9c3c8a53c1c6bfd0f3856f4c3f"
server_key_iv: "80168b1c6e8f9da1cec3c0739a928654"
server_key_version: "encryptionkey@0"
I20260811 19:16:55.554980 4473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/master-0/wal/instance:
uuid: "41c487474a9b46629ca620a804fb5ff8"
format_stamp: "Formatted at 2026-08-11 19:16:55 on dist-test-slave-t7g5"
server_key: "b6f2ef9c3c8a53c1c6bfd0f3856f4c3f"
server_key_iv: "80168b1c6e8f9da1cec3c0739a928654"
server_key_version: "encryptionkey@0"
I20260811 19:16:55.561021 4473 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260811 19:16:55.564944 4494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:55.566670 4473 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 19:16:55.566886 4473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/master-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/master-0/wal
uuid: "41c487474a9b46629ca620a804fb5ff8"
format_stamp: "Formatted at 2026-08-11 19:16:55 on dist-test-slave-t7g5"
server_key: "b6f2ef9c3c8a53c1c6bfd0f3856f4c3f"
server_key_iv: "80168b1c6e8f9da1cec3c0739a928654"
server_key_version: "encryptionkey@0"
I20260811 19:16:55.567121 4473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:16:55.589644 4473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:16:55.590643 4473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:16:55.590955 4473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:16:55.611306 4473 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.190:44105
I20260811 19:16:55.611354 4596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.190:44105 every 8 connection(s)
I20260811 19:16:55.612740 4473 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
I20260811 19:16:55.616883 4597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:16:55.619789 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 4473
I20260811 19:16:55.619961 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/master-0/wal/instance
I20260811 19:16:55.620394 3318 external_mini_cluster.cc:1428] Setting key 9cd8c5b616a079ebec95fad9af456615
I20260811 19:16:55.630082 4597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41c487474a9b46629ca620a804fb5ff8: Bootstrap starting.
I20260811 19:16:55.633399 4597 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 41c487474a9b46629ca620a804fb5ff8: Neither blocks nor log segments found. Creating new log.
I20260811 19:16:55.634641 4597 log.cc:824] T 00000000000000000000000000000000 P 41c487474a9b46629ca620a804fb5ff8: Log is configured to *not* fsync() on all Append() calls
I20260811 19:16:55.635355 4604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:55.622610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45752 (local address 127.3.61.190:44105)
0811 19:16:55.623835 (+ 1225us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:55.623847 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:55.623903 (+ 56us) server_negotiation.cc:407] Connection header received
0811 19:16:55.625748 (+ 1845us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:55.625775 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:55.626344 (+ 569us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:55.626743 (+ 399us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:16:55.627368 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:55.628713 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:55.630114 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:55.630874 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:55.631052 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:55.632227 (+ 1175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:16:55.632234 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:16:55.632249 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:16:55.632803 (+ 554us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:16:55.632905 (+ 102us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:16:55.632927 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:16:55.633032 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:55.633290 (+ 258us) server_negotiation.cc:299] Negotiation successful
0811 19:16:55.633771 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":676,"thread_start_us":413,"threads_started":1}
I20260811 19:16:55.638032 4597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41c487474a9b46629ca620a804fb5ff8: No bootstrap required, opened a new log
I20260811 19:16:55.647202 4597 raft_consensus.cc:348] T 00000000000000000000000000000000 P 41c487474a9b46629ca620a804fb5ff8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41c487474a9b46629ca620a804fb5ff8" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 44105 } }
I20260811 19:16:55.647605 4597 raft_consensus.cc:374] T 00000000000000000000000000000000 P 41c487474a9b46629ca620a804fb5ff8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:16:55.647794 4597 raft_consensus.cc:729] T 00000000000000000000000000000000 P 41c487474a9b46629ca620a804fb5ff8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41c487474a9b46629ca620a804fb5ff8, State: Initialized, Role: FOLLOWER
I20260811 19:16:55.648541 4597 consensus_queue.cc:260] T 00000000000000000000000000000000 P 41c487474a9b46629ca620a804fb5ff8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41c487474a9b46629ca620a804fb5ff8" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 44105 } }
I20260811 19:16:55.648859 4597 raft_consensus.cc:388] T 00000000000000000000000000000000 P 41c487474a9b46629ca620a804fb5ff8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:16:55.649015 4597 raft_consensus.cc:482] T 00000000000000000000000000000000 P 41c487474a9b46629ca620a804fb5ff8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:16:55.649214 4597 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 41c487474a9b46629ca620a804fb5ff8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:16:55.651520 4597 raft_consensus.cc:504] T 00000000000000000000000000000000 P 41c487474a9b46629ca620a804fb5ff8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41c487474a9b46629ca620a804fb5ff8" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 44105 } }
I20260811 19:16:55.652070 4597 leader_election.cc:302] T 00000000000000000000000000000000 P 41c487474a9b46629ca620a804fb5ff8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 41c487474a9b46629ca620a804fb5ff8; no voters:
I20260811 19:16:55.652745 4597 leader_election.cc:288] T 00000000000000000000000000000000 P 41c487474a9b46629ca620a804fb5ff8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:16:55.652889 4607 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 41c487474a9b46629ca620a804fb5ff8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:16:55.653609 4607 raft_consensus.cc:686] T 00000000000000000000000000000000 P 41c487474a9b46629ca620a804fb5ff8 [term 1 LEADER]: Becoming Leader. State: Replica: 41c487474a9b46629ca620a804fb5ff8, State: Running, Role: LEADER
I20260811 19:16:55.654155 4607 consensus_queue.cc:237] T 00000000000000000000000000000000 P 41c487474a9b46629ca620a804fb5ff8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41c487474a9b46629ca620a804fb5ff8" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 44105 } }
I20260811 19:16:55.654899 4597 sys_catalog.cc:564] T 00000000000000000000000000000000 P 41c487474a9b46629ca620a804fb5ff8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:16:55.656919 4611 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41c487474a9b46629ca620a804fb5ff8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 41c487474a9b46629ca620a804fb5ff8. Latest consensus state: current_term: 1 leader_uuid: "41c487474a9b46629ca620a804fb5ff8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41c487474a9b46629ca620a804fb5ff8" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 44105 } } }
I20260811 19:16:55.657311 4611 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41c487474a9b46629ca620a804fb5ff8 [sys.catalog]: This master's current role is: LEADER
I20260811 19:16:55.657855 4608 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41c487474a9b46629ca620a804fb5ff8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "41c487474a9b46629ca620a804fb5ff8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41c487474a9b46629ca620a804fb5ff8" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 44105 } } }
I20260811 19:16:55.658164 4608 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41c487474a9b46629ca620a804fb5ff8 [sys.catalog]: This master's current role is: LEADER
I20260811 19:16:55.660836 4626 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:16:55.668203 4626 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:16:55.678177 4626 catalog_manager.cc:1347] Generated new cluster ID: 34771387e79942768d13c6d2d2e9a540
I20260811 19:16:55.678319 4626 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:16:55.690763 4626 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:16:55.692173 4626 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:16:55.698114 4626 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 41c487474a9b46629ca620a804fb5ff8: Generated new TSK 0
I20260811 19:16:55.699065 4626 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:16:55.713344 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.129:0
--local_ip_for_outbound_sockets=127.3.61.129
--webserver_interface=127.3.61.129
--webserver_port=0
--tserver_master_addrs=127.3.61.190:44105
--builtin_ntp_servers=127.3.61.148:39273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 19:16:56.118721 4640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:16:56.119091 4640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:16:56.119246 4640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:16:56.128240 4640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:16:56.128432 4640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:16:56.128611 4640 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.129
I20260811 19:16:56.140285 4640 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:39273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.61.129
--webserver_port=0
--tserver_master_addrs=127.3.61.190:44105
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.129
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:16:56.141800 4640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:16:56.143738 4640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:16:56.160734 4650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:16:56.160760 4654 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:16:56.161000 4651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:16:56.162201 4640 server_base.cc:1034] running on GCE node
I20260811 19:16:56.162961 4640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:16:56.164165 4640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:16:56.165516 4640 hybrid_clock.cc:648] HybridClock initialized: now 1786475816165429 us; error 38 us; skew 500 ppm
I20260811 19:16:56.166136 4640 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 19:16:56.169263 4640 webserver.cc:460] Webserver started at http://127.3.61.129:38901/ using document root <none> and password file <none>
I20260811 19:16:56.170151 4640 fs_manager.cc:359] Metadata directory not provided
I20260811 19:16:56.170266 4640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:16:56.170605 4640 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:16:56.173928 4640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/ts-0/data/instance:
uuid: "45ba5cdaa5f8446e947e5d13b56171bf"
format_stamp: "Formatted at 2026-08-11 19:16:56 on dist-test-slave-t7g5"
server_key: "27f5aadf6b0afbfb30cdaf1aceba0b47"
server_key_iv: "1da6c9dd9e3d533f3bca5237642714cf"
server_key_version: "encryptionkey@0"
I20260811 19:16:56.174767 4640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/ts-0/wal/instance:
uuid: "45ba5cdaa5f8446e947e5d13b56171bf"
format_stamp: "Formatted at 2026-08-11 19:16:56 on dist-test-slave-t7g5"
server_key: "27f5aadf6b0afbfb30cdaf1aceba0b47"
server_key_iv: "1da6c9dd9e3d533f3bca5237642714cf"
server_key_version: "encryptionkey@0"
I20260811 19:16:56.181018 4640 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260811 19:16:56.185436 4661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:56.187619 4640 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 19:16:56.187856 4640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/ts-0/wal
uuid: "45ba5cdaa5f8446e947e5d13b56171bf"
format_stamp: "Formatted at 2026-08-11 19:16:56 on dist-test-slave-t7g5"
server_key: "27f5aadf6b0afbfb30cdaf1aceba0b47"
server_key_iv: "1da6c9dd9e3d533f3bca5237642714cf"
server_key_version: "encryptionkey@0"
I20260811 19:16:56.188107 4640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:16:56.208909 4640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:16:56.209929 4640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:16:56.210243 4640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:16:56.211373 4640 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:16:56.212996 4640 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:16:56.213099 4640 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:56.213192 4640 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:16:56.213274 4640 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:56.249670 4640 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.129:43271
I20260811 19:16:56.249768 4861 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.129:43271 every 8 connection(s)
I20260811 19:16:56.251071 4640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/ts-0/data/info.pb
I20260811 19:16:56.259622 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 4640
I20260811 19:16:56.259827 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/ts-0/wal/instance
I20260811 19:16:56.260445 3318 external_mini_cluster.cc:1428] Setting key 0ddf80f54120d1d11ae78530e490216d
I20260811 19:16:56.263729 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.130:0
--local_ip_for_outbound_sockets=127.3.61.130
--webserver_interface=127.3.61.130
--webserver_port=0
--tserver_master_addrs=127.3.61.190:44105
--builtin_ntp_servers=127.3.61.148:39273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 19:16:56.268335 4866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:56.254299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:38713 (local address 127.3.61.190:44105)
0811 19:16:56.254841 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:56.254847 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:56.256582 (+ 1735us) server_negotiation.cc:407] Connection header received
0811 19:16:56.258639 (+ 2057us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:56.258645 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:56.258736 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:56.258845 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:16:56.260632 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:56.261591 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:56.264450 (+ 2859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:56.264880 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:56.265005 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:56.267371 (+ 2366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:16:56.267379 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:16:56.267383 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:16:56.267406 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:16:56.267467 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:16:56.267480 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:16:56.267570 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:56.267928 (+ 358us) server_negotiation.cc:299] Negotiation successful
0811 19:16:56.268050 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":284,"threads_started":1}
I20260811 19:16:56.270205 4867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:56.254997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:44105 (local address 127.3.61.129:38713)
0811 19:16:56.256390 (+ 1393us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:56.256439 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 19:16:56.258318 (+ 1879us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:16:56.259160 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:16:56.259174 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:16:56.259417 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:16:56.260355 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:56.260396 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:56.261811 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:56.261816 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:56.264161 (+ 2345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:56.264200 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:56.265220 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:56.265230 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:56.265725 (+ 495us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:56.266797 (+ 1072us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 19:16:56.266801 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:16:56.266890 (+ 89us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 19:16:56.267043 (+ 153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:16:56.267606 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:16:56.267618 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:16:56.267628 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 19:16:56.267869 (+ 241us) client_negotiation.cc:240] Negotiation successful
0811 19:16:56.268446 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":739,"thread_start_us":432,"threads_started":1}
I20260811 19:16:56.272691 4864 heartbeater.cc:347] Connected to a master server at 127.3.61.190:44105
I20260811 19:16:56.273058 4864 heartbeater.cc:464] Registering TS with master...
I20260811 19:16:56.274024 4864 heartbeater.cc:511] Master 127.3.61.190:44105 requested a full tablet report, sending...
I20260811 19:16:56.276783 4529 ts_manager.cc:194] Registered new tserver with Master: 45ba5cdaa5f8446e947e5d13b56171bf (127.3.61.129:43271)
I20260811 19:16:56.278383 4529 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.61.129:38713
W20260811 19:16:56.677471 4869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:16:56.677883 4869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:16:56.678007 4869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:16:56.686985 4869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:16:56.687139 4869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:16:56.687292 4869 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.130
I20260811 19:16:56.699627 4869 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:39273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.61.130
--webserver_port=0
--tserver_master_addrs=127.3.61.190:44105
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.130
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:16:56.701114 4869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:16:56.703249 4869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:16:56.718927 4878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:16:56.718887 4881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:16:56.718951 4877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:16:56.724089 4869 server_base.cc:1034] running on GCE node
I20260811 19:16:56.725324 4869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:16:56.726802 4869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:16:56.728132 4869 hybrid_clock.cc:648] HybridClock initialized: now 1786475816728070 us; error 57 us; skew 500 ppm
I20260811 19:16:56.728695 4869 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 19:16:56.732486 4869 webserver.cc:460] Webserver started at http://127.3.61.130:45275/ using document root <none> and password file <none>
I20260811 19:16:56.733635 4869 fs_manager.cc:359] Metadata directory not provided
I20260811 19:16:56.733851 4869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:16:56.734270 4869 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:16:56.737919 4869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/ts-1/data/instance:
uuid: "40c9c1ee2ba745dfb7225f98dc620e47"
format_stamp: "Formatted at 2026-08-11 19:16:56 on dist-test-slave-t7g5"
server_key: "19116d5b50c8c7c7d7b1cc038f4438d8"
server_key_iv: "eaeaaeb2b03dfcf7eb22315be8633532"
server_key_version: "encryptionkey@0"
I20260811 19:16:56.738946 4869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/ts-1/wal/instance:
uuid: "40c9c1ee2ba745dfb7225f98dc620e47"
format_stamp: "Formatted at 2026-08-11 19:16:56 on dist-test-slave-t7g5"
server_key: "19116d5b50c8c7c7d7b1cc038f4438d8"
server_key_iv: "eaeaaeb2b03dfcf7eb22315be8633532"
server_key_version: "encryptionkey@0"
I20260811 19:16:56.747864 4869 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.005s sys 0.003s
I20260811 19:16:56.754114 4889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:56.756207 4869 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.002s
I20260811 19:16:56.756433 4869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/ts-1/wal
uuid: "40c9c1ee2ba745dfb7225f98dc620e47"
format_stamp: "Formatted at 2026-08-11 19:16:56 on dist-test-slave-t7g5"
server_key: "19116d5b50c8c7c7d7b1cc038f4438d8"
server_key_iv: "eaeaaeb2b03dfcf7eb22315be8633532"
server_key_version: "encryptionkey@0"
I20260811 19:16:56.756699 4869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:16:56.776567 4869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:16:56.777580 4869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:16:56.777935 4869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:16:56.779150 4869 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:16:56.781051 4869 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:16:56.781162 4869 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:56.781288 4869 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:16:56.781360 4869 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:56.822108 4869 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.130:45741
I20260811 19:16:56.822214 5087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.130:45741 every 8 connection(s)
I20260811 19:16:56.824044 4869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/ts-1/data/info.pb
I20260811 19:16:56.828619 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 4869
I20260811 19:16:56.828891 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/ts-1/wal/instance
I20260811 19:16:56.829824 3318 external_mini_cluster.cc:1428] Setting key 333b47717ae2ededfd9be629a56e12f2
I20260811 19:16:56.833274 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.61.131:0
--local_ip_for_outbound_sockets=127.3.61.131
--webserver_interface=127.3.61.131
--webserver_port=0
--tserver_master_addrs=127.3.61.190:44105
--builtin_ntp_servers=127.3.61.148:39273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 19:16:56.841960 5095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:56.827497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.130:51643 (local address 127.3.61.190:44105)
0811 19:16:56.828156 (+ 659us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:56.828164 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:56.830323 (+ 2159us) server_negotiation.cc:407] Connection header received
0811 19:16:56.832713 (+ 2390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:56.832720 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:56.832829 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:56.832948 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:16:56.834680 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:56.835597 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:56.838307 (+ 2710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:56.838808 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:56.838957 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:56.840899 (+ 1942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:16:56.840906 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:16:56.840910 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:16:56.840937 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:16:56.840984 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:16:56.840998 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:16:56.841082 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:56.841355 (+ 273us) server_negotiation.cc:299] Negotiation successful
0811 19:16:56.841502 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":527,"thread_start_us":325,"threads_started":1}
I20260811 19:16:56.843199 5096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:56.828215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:44105 (local address 127.3.61.130:51643)
0811 19:16:56.830079 (+ 1864us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:56.830154 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0811 19:16:56.832314 (+ 2160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:16:56.833307 (+ 993us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:16:56.833320 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:16:56.833483 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:16:56.834428 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:56.834453 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:56.835790 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:56.835795 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:56.837939 (+ 2144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:56.837975 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:56.839222 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:56.839241 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:56.839546 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:56.840409 (+ 863us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 19:16:56.840412 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:16:56.840466 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 19:16:56.840575 (+ 109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:16:56.841104 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:16:56.841116 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:16:56.841125 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 19:16:56.841306 (+ 181us) client_negotiation.cc:240] Negotiation successful
0811 19:16:56.841876 (+ 570us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":995,"thread_start_us":712,"threads_started":1}
I20260811 19:16:56.845317 5088 heartbeater.cc:347] Connected to a master server at 127.3.61.190:44105
I20260811 19:16:56.845925 5088 heartbeater.cc:464] Registering TS with master...
I20260811 19:16:56.846993 5088 heartbeater.cc:511] Master 127.3.61.190:44105 requested a full tablet report, sending...
I20260811 19:16:56.849354 4529 ts_manager.cc:194] Registered new tserver with Master: 40c9c1ee2ba745dfb7225f98dc620e47 (127.3.61.130:45741)
I20260811 19:16:56.850935 4529 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.61.130:51643
W20260811 19:16:57.274796 5097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:16:57.275157 5097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:16:57.275265 5097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 19:16:57.281960 4864 heartbeater.cc:503] Master 127.3.61.190:44105 was elected leader, sending a full tablet report...
W20260811 19:16:57.284071 5097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:16:57.284240 5097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:16:57.284431 5097 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.61.131
I20260811 19:16:57.296394 5097 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:39273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.61.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.61.131
--webserver_port=0
--tserver_master_addrs=127.3.61.190:44105
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.61.131
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:16:57.298101 5097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:16:57.300313 5097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:16:57.317414 5104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:16:57.317492 5106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:16:57.317474 5110 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:16:57.319240 5097 server_base.cc:1034] running on GCE node
I20260811 19:16:57.320065 5097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:16:57.321349 5097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:16:57.322710 5097 hybrid_clock.cc:648] HybridClock initialized: now 1786475817322609 us; error 62 us; skew 500 ppm
I20260811 19:16:57.323280 5097 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 19:16:57.330538 5097 webserver.cc:460] Webserver started at http://127.3.61.131:44767/ using document root <none> and password file <none>
I20260811 19:16:57.331884 5097 fs_manager.cc:359] Metadata directory not provided
I20260811 19:16:57.332096 5097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:16:57.332568 5097 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:16:57.337062 5097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/ts-2/data/instance:
uuid: "44a8c55066964a32a0bae081e3774aaa"
format_stamp: "Formatted at 2026-08-11 19:16:57 on dist-test-slave-t7g5"
server_key: "f2b732d619d348f890d7b4a616d7d771"
server_key_iv: "45620591ada63c0ad19bdd92e3f140f1"
server_key_version: "encryptionkey@0"
I20260811 19:16:57.338430 5097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/ts-2/wal/instance:
uuid: "44a8c55066964a32a0bae081e3774aaa"
format_stamp: "Formatted at 2026-08-11 19:16:57 on dist-test-slave-t7g5"
server_key: "f2b732d619d348f890d7b4a616d7d771"
server_key_iv: "45620591ada63c0ad19bdd92e3f140f1"
server_key_version: "encryptionkey@0"
I20260811 19:16:57.345605 5097 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260811 19:16:57.350401 5118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:57.352849 5097 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260811 19:16:57.353089 5097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/ts-2/wal
uuid: "44a8c55066964a32a0bae081e3774aaa"
format_stamp: "Formatted at 2026-08-11 19:16:57 on dist-test-slave-t7g5"
server_key: "f2b732d619d348f890d7b4a616d7d771"
server_key_iv: "45620591ada63c0ad19bdd92e3f140f1"
server_key_version: "encryptionkey@0"
I20260811 19:16:57.353372 5097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:16:57.398523 5097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:16:57.399529 5097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:16:57.399874 5097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:16:57.401327 5097 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 19:16:57.403148 5097 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 19:16:57.403257 5097 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:57.403354 5097 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 19:16:57.403434 5097 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 19:16:57.442662 5097 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.131:39339
I20260811 19:16:57.442776 5300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.131:39339 every 8 connection(s)
I20260811 19:16:57.444156 5097 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/ts-2/data/info.pb
I20260811 19:16:57.453614 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 5097
I20260811 19:16:57.453830 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/ts-2/wal/instance
I20260811 19:16:57.454273 3318 external_mini_cluster.cc:1428] Setting key d89d18fc33f962d2bafd9e8c3cfdfd5b
I20260811 19:16:57.459947 5305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:57.447666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.131:37527 (local address 127.3.61.190:44105)
0811 19:16:57.448328 (+ 662us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:57.448334 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:57.449862 (+ 1528us) server_negotiation.cc:407] Connection header received
0811 19:16:57.451901 (+ 2039us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:57.451907 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:57.452008 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:57.452116 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:16:57.453569 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:57.454506 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:57.456710 (+ 2204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:57.457149 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:57.457266 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:57.459004 (+ 1738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:16:57.459011 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:16:57.459017 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:16:57.459042 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:16:57.459101 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:16:57.459118 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:16:57.459214 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:57.459511 (+ 297us) server_negotiation.cc:299] Negotiation successful
0811 19:16:57.459630 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":523,"thread_start_us":295,"threads_started":1}
I20260811 19:16:57.461052 5307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:57.448180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:44105 (local address 127.3.61.131:37527)
0811 19:16:57.449528 (+ 1348us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:57.449706 (+ 178us) client_negotiation.cc:174] Beginning negotiation
0811 19:16:57.451574 (+ 1868us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:16:57.452403 (+ 829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:16:57.452413 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:16:57.452518 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:16:57.453349 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:57.453371 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:57.454684 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:57.454690 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:57.456566 (+ 1876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:57.456584 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:57.457396 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:57.457406 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:57.457755 (+ 349us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:57.458567 (+ 812us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 19:16:57.458571 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:16:57.458657 (+ 86us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 19:16:57.458781 (+ 124us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:16:57.459229 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:16:57.459241 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:16:57.459250 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 19:16:57.459472 (+ 222us) client_negotiation.cc:240] Negotiation successful
0811 19:16:57.459883 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"thread_start_us":396,"threads_started":1}
I20260811 19:16:57.463203 5303 heartbeater.cc:347] Connected to a master server at 127.3.61.190:44105
I20260811 19:16:57.463600 5303 heartbeater.cc:464] Registering TS with master...
I20260811 19:16:57.464463 5303 heartbeater.cc:511] Master 127.3.61.190:44105 requested a full tablet report, sending...
I20260811 19:16:57.466373 4529 ts_manager.cc:194] Registered new tserver with Master: 44a8c55066964a32a0bae081e3774aaa (127.3.61.131:39339)
I20260811 19:16:57.467258 4529 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.61.131:37527
I20260811 19:16:57.471486 3318 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 19:16:57.484076 5305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:57.476540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45768 (local address 127.3.61.190:44105)
0811 19:16:57.476853 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:57.476861 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:57.477199 (+ 338us) server_negotiation.cc:407] Connection header received
0811 19:16:57.477414 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:57.477418 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:57.477487 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:57.477578 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:16:57.478135 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:57.479135 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:57.480866 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:57.481363 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:57.481501 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:57.483016 (+ 1515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:16:57.483021 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:16:57.483026 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:16:57.483054 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:16:57.483122 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:16:57.483137 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:16:57.483227 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:57.483373 (+ 146us) server_negotiation.cc:299] Negotiation successful
0811 19:16:57.483508 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128}
I20260811 19:16:57.489992 4529 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45768:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 19:16:57.499195 4529 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 19:16:57.529467 5328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:57.517300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36972 (local address 127.3.61.130:45741)
0811 19:16:57.518031 (+ 731us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:57.518038 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:57.520590 (+ 2552us) server_negotiation.cc:407] Connection header received
0811 19:16:57.520744 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:57.520751 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:57.520949 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:57.521124 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:16:57.522079 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:57.523428 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:57.526207 (+ 2779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:57.527848 (+ 1641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:57.528015 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:57.528313 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:57.528810 (+ 497us) server_negotiation.cc:299] Negotiation successful
0811 19:16:57.529068 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":1080,"threads_started":1}
I20260811 19:16:57.529496 5330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:57.517605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.130:45741 (local address 127.0.0.1:36972)
0811 19:16:57.520430 (+ 2825us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:57.520453 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 19:16:57.520597 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:16:57.521385 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:16:57.521388 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:16:57.521412 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:16:57.521866 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:57.521881 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:57.523613 (+ 1732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:57.523622 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:57.525900 (+ 2278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:57.525918 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:57.528124 (+ 2206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:57.528131 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:57.528409 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:57.528476 (+ 67us) client_negotiation.cc:769] Sending connection context
0811 19:16:57.528657 (+ 181us) client_negotiation.cc:240] Negotiation successful
0811 19:16:57.528957 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2690,"mutex_wait_us":3,"thread_start_us":2516,"threads_started":1}
I20260811 19:16:57.529692 5323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:57.516379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.131:39339 (local address 127.0.0.1:39130)
0811 19:16:57.518447 (+ 2068us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:57.518506 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 19:16:57.518745 (+ 239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:16:57.520012 (+ 1267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:16:57.520021 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:16:57.520069 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:16:57.520585 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:57.520597 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:57.522627 (+ 2030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:57.522632 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:57.525264 (+ 2632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:57.525285 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:57.528537 (+ 3252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:57.528545 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:57.528761 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:57.528808 (+ 47us) client_negotiation.cc:769] Sending connection context
0811 19:16:57.528920 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 19:16:57.529097 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1739,"spinlock_wait_cycles":12032,"thread_start_us":1089,"threads_started":1}
I20260811 19:16:57.530829 5327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:57.516819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52608 (local address 127.3.61.129:43271)
0811 19:16:57.518488 (+ 1669us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:57.518496 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:57.520427 (+ 1931us) server_negotiation.cc:407] Connection header received
0811 19:16:57.520553 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:57.520561 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:57.520830 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:57.521072 (+ 242us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:16:57.521853 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:57.523394 (+ 1541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:57.527436 (+ 4042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:57.529046 (+ 1610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:57.529185 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:57.529502 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:57.530131 (+ 629us) server_negotiation.cc:299] Negotiation successful
0811 19:16:57.530364 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1484,"thread_start_us":1337,"threads_started":1}
I20260811 19:16:57.530902 5325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:57.516643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39130 (local address 127.3.61.131:39339)
0811 19:16:57.517577 (+ 934us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:57.517586 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:57.518640 (+ 1054us) server_negotiation.cc:407] Connection header received
0811 19:16:57.519083 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:57.519095 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:57.519427 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:57.519652 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:16:57.520799 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:57.522449 (+ 1650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:57.525590 (+ 3141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:57.527913 (+ 2323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:57.528102 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:57.528491 (+ 389us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:57.529927 (+ 1436us) server_negotiation.cc:299] Negotiation successful
0811 19:16:57.530179 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":747,"thread_start_us":578,"threads_started":1}
I20260811 19:16:57.531553 5331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:57.516954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:43271 (local address 127.0.0.1:52608)
0811 19:16:57.520078 (+ 3124us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:57.520105 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 19:16:57.520294 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:16:57.521071 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:16:57.521075 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:16:57.521101 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:16:57.521576 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:57.521589 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:57.523556 (+ 1967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:57.523565 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:57.527049 (+ 3484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:57.527087 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:57.529694 (+ 2607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:57.529704 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:57.529902 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:57.529935 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 19:16:57.530023 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 19:16:57.530170 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2957,"mutex_wait_us":246,"spinlock_wait_cycles":896,"thread_start_us":2775,"threads_started":1}
I20260811 19:16:57.535108 4969 tablet_service.cc:1467] Processing CreateTablet for tablet 48584db90ff642d59db61b2a91fcf968 (DEFAULT_TABLE table=test-table [id=dea961f2c33d46a89778704da0db0db1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:16:57.535558 5188 tablet_service.cc:1467] Processing CreateTablet for tablet 48584db90ff642d59db61b2a91fcf968 (DEFAULT_TABLE table=test-table [id=dea961f2c33d46a89778704da0db0db1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:16:57.536605 4969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48584db90ff642d59db61b2a91fcf968. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:16:57.537101 5188 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48584db90ff642d59db61b2a91fcf968. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:16:57.536725 4744 tablet_service.cc:1467] Processing CreateTablet for tablet 48584db90ff642d59db61b2a91fcf968 (DEFAULT_TABLE table=test-table [id=dea961f2c33d46a89778704da0db0db1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 19:16:57.538193 4744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48584db90ff642d59db61b2a91fcf968. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:16:57.550020 5335 tablet_bootstrap.cc:492] T 48584db90ff642d59db61b2a91fcf968 P 40c9c1ee2ba745dfb7225f98dc620e47: Bootstrap starting.
I20260811 19:16:57.552876 5338 tablet_bootstrap.cc:492] T 48584db90ff642d59db61b2a91fcf968 P 45ba5cdaa5f8446e947e5d13b56171bf: Bootstrap starting.
I20260811 19:16:57.554626 5337 tablet_bootstrap.cc:492] T 48584db90ff642d59db61b2a91fcf968 P 44a8c55066964a32a0bae081e3774aaa: Bootstrap starting.
I20260811 19:16:57.555325 5335 tablet_bootstrap.cc:654] T 48584db90ff642d59db61b2a91fcf968 P 40c9c1ee2ba745dfb7225f98dc620e47: Neither blocks nor log segments found. Creating new log.
I20260811 19:16:57.556787 5335 log.cc:824] T 48584db90ff642d59db61b2a91fcf968 P 40c9c1ee2ba745dfb7225f98dc620e47: Log is configured to *not* fsync() on all Append() calls
I20260811 19:16:57.557032 5338 tablet_bootstrap.cc:654] T 48584db90ff642d59db61b2a91fcf968 P 45ba5cdaa5f8446e947e5d13b56171bf: Neither blocks nor log segments found. Creating new log.
I20260811 19:16:57.558163 5337 tablet_bootstrap.cc:654] T 48584db90ff642d59db61b2a91fcf968 P 44a8c55066964a32a0bae081e3774aaa: Neither blocks nor log segments found. Creating new log.
I20260811 19:16:57.558408 5338 log.cc:824] T 48584db90ff642d59db61b2a91fcf968 P 45ba5cdaa5f8446e947e5d13b56171bf: Log is configured to *not* fsync() on all Append() calls
I20260811 19:16:57.559409 5337 log.cc:824] T 48584db90ff642d59db61b2a91fcf968 P 44a8c55066964a32a0bae081e3774aaa: Log is configured to *not* fsync() on all Append() calls
I20260811 19:16:57.560724 5335 tablet_bootstrap.cc:492] T 48584db90ff642d59db61b2a91fcf968 P 40c9c1ee2ba745dfb7225f98dc620e47: No bootstrap required, opened a new log
I20260811 19:16:57.561064 5335 ts_tablet_manager.cc:1397] T 48584db90ff642d59db61b2a91fcf968 P 40c9c1ee2ba745dfb7225f98dc620e47: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260811 19:16:57.563052 5338 tablet_bootstrap.cc:492] T 48584db90ff642d59db61b2a91fcf968 P 45ba5cdaa5f8446e947e5d13b56171bf: No bootstrap required, opened a new log
I20260811 19:16:57.563370 5338 ts_tablet_manager.cc:1397] T 48584db90ff642d59db61b2a91fcf968 P 45ba5cdaa5f8446e947e5d13b56171bf: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260811 19:16:57.569310 5337 tablet_bootstrap.cc:492] T 48584db90ff642d59db61b2a91fcf968 P 44a8c55066964a32a0bae081e3774aaa: No bootstrap required, opened a new log
I20260811 19:16:57.569819 5337 ts_tablet_manager.cc:1397] T 48584db90ff642d59db61b2a91fcf968 P 44a8c55066964a32a0bae081e3774aaa: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.008s
I20260811 19:16:57.574150 5335 raft_consensus.cc:348] T 48584db90ff642d59db61b2a91fcf968 P 40c9c1ee2ba745dfb7225f98dc620e47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44a8c55066964a32a0bae081e3774aaa" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39339 } } peers { permanent_uuid: "45ba5cdaa5f8446e947e5d13b56171bf" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 43271 } } peers { permanent_uuid: "40c9c1ee2ba745dfb7225f98dc620e47" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45741 } }
I20260811 19:16:57.574752 5335 raft_consensus.cc:374] T 48584db90ff642d59db61b2a91fcf968 P 40c9c1ee2ba745dfb7225f98dc620e47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:16:57.574875 5335 raft_consensus.cc:729] T 48584db90ff642d59db61b2a91fcf968 P 40c9c1ee2ba745dfb7225f98dc620e47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40c9c1ee2ba745dfb7225f98dc620e47, State: Initialized, Role: FOLLOWER
I20260811 19:16:57.575699 5335 consensus_queue.cc:260] T 48584db90ff642d59db61b2a91fcf968 P 40c9c1ee2ba745dfb7225f98dc620e47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44a8c55066964a32a0bae081e3774aaa" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39339 } } peers { permanent_uuid: "45ba5cdaa5f8446e947e5d13b56171bf" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 43271 } } peers { permanent_uuid: "40c9c1ee2ba745dfb7225f98dc620e47" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45741 } }
I20260811 19:16:57.576268 5338 raft_consensus.cc:348] T 48584db90ff642d59db61b2a91fcf968 P 45ba5cdaa5f8446e947e5d13b56171bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44a8c55066964a32a0bae081e3774aaa" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39339 } } peers { permanent_uuid: "45ba5cdaa5f8446e947e5d13b56171bf" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 43271 } } peers { permanent_uuid: "40c9c1ee2ba745dfb7225f98dc620e47" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45741 } }
I20260811 19:16:57.576823 5338 raft_consensus.cc:374] T 48584db90ff642d59db61b2a91fcf968 P 45ba5cdaa5f8446e947e5d13b56171bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:16:57.576944 5338 raft_consensus.cc:729] T 48584db90ff642d59db61b2a91fcf968 P 45ba5cdaa5f8446e947e5d13b56171bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45ba5cdaa5f8446e947e5d13b56171bf, State: Initialized, Role: FOLLOWER
I20260811 19:16:57.577775 5338 consensus_queue.cc:260] T 48584db90ff642d59db61b2a91fcf968 P 45ba5cdaa5f8446e947e5d13b56171bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44a8c55066964a32a0bae081e3774aaa" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39339 } } peers { permanent_uuid: "45ba5cdaa5f8446e947e5d13b56171bf" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 43271 } } peers { permanent_uuid: "40c9c1ee2ba745dfb7225f98dc620e47" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45741 } }
I20260811 19:16:57.578835 5088 heartbeater.cc:503] Master 127.3.61.190:44105 was elected leader, sending a full tablet report...
I20260811 19:16:57.579403 5335 ts_tablet_manager.cc:1428] T 48584db90ff642d59db61b2a91fcf968 P 40c9c1ee2ba745dfb7225f98dc620e47: Time spent starting tablet: real 0.018s user 0.015s sys 0.004s
I20260811 19:16:57.580892 5337 raft_consensus.cc:348] T 48584db90ff642d59db61b2a91fcf968 P 44a8c55066964a32a0bae081e3774aaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44a8c55066964a32a0bae081e3774aaa" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39339 } } peers { permanent_uuid: "45ba5cdaa5f8446e947e5d13b56171bf" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 43271 } } peers { permanent_uuid: "40c9c1ee2ba745dfb7225f98dc620e47" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45741 } }
I20260811 19:16:57.581437 5337 raft_consensus.cc:374] T 48584db90ff642d59db61b2a91fcf968 P 44a8c55066964a32a0bae081e3774aaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:16:57.581549 5337 raft_consensus.cc:729] T 48584db90ff642d59db61b2a91fcf968 P 44a8c55066964a32a0bae081e3774aaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44a8c55066964a32a0bae081e3774aaa, State: Initialized, Role: FOLLOWER
I20260811 19:16:57.582261 5337 consensus_queue.cc:260] T 48584db90ff642d59db61b2a91fcf968 P 44a8c55066964a32a0bae081e3774aaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44a8c55066964a32a0bae081e3774aaa" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39339 } } peers { permanent_uuid: "45ba5cdaa5f8446e947e5d13b56171bf" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 43271 } } peers { permanent_uuid: "40c9c1ee2ba745dfb7225f98dc620e47" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45741 } }
I20260811 19:16:57.583642 5303 heartbeater.cc:503] Master 127.3.61.190:44105 was elected leader, sending a full tablet report...
I20260811 19:16:57.584373 5337 ts_tablet_manager.cc:1428] T 48584db90ff642d59db61b2a91fcf968 P 44a8c55066964a32a0bae081e3774aaa: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260811 19:16:57.586354 5338 ts_tablet_manager.cc:1428] T 48584db90ff642d59db61b2a91fcf968 P 45ba5cdaa5f8446e947e5d13b56171bf: Time spent starting tablet: real 0.023s user 0.019s sys 0.004s
I20260811 19:16:57.692607 5349 raft_consensus.cc:482] T 48584db90ff642d59db61b2a91fcf968 P 45ba5cdaa5f8446e947e5d13b56171bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 19:16:57.692957 5349 raft_consensus.cc:504] T 48584db90ff642d59db61b2a91fcf968 P 45ba5cdaa5f8446e947e5d13b56171bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44a8c55066964a32a0bae081e3774aaa" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39339 } } peers { permanent_uuid: "45ba5cdaa5f8446e947e5d13b56171bf" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 43271 } } peers { permanent_uuid: "40c9c1ee2ba745dfb7225f98dc620e47" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45741 } }
I20260811 19:16:57.694764 5349 leader_election.cc:288] T 48584db90ff642d59db61b2a91fcf968 P 45ba5cdaa5f8446e947e5d13b56171bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 44a8c55066964a32a0bae081e3774aaa (127.3.61.131:39339), 40c9c1ee2ba745dfb7225f98dc620e47 (127.3.61.130:45741)
W20260811 19:16:57.696053 5304 tablet.cc:2369] T 48584db90ff642d59db61b2a91fcf968 P 44a8c55066964a32a0bae081e3774aaa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:16:57.702880 5325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:57.695284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:54709 (local address 127.3.61.131:39339)
0811 19:16:57.695546 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:57.695553 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:57.696238 (+ 685us) server_negotiation.cc:407] Connection header received
0811 19:16:57.696634 (+ 396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:57.696640 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:57.696726 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:57.696847 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:16:57.697728 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:57.698461 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:57.700916 (+ 2455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:57.701853 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:57.702034 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:57.702124 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:57.702456 (+ 332us) server_negotiation.cc:299] Negotiation successful
0811 19:16:57.702589 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130}
I20260811 19:16:57.702991 5328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:57.695277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.61.129:37739 (local address 127.3.61.130:45741)
0811 19:16:57.695546 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:57.695553 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:57.696282 (+ 729us) server_negotiation.cc:407] Connection header received
0811 19:16:57.696555 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:57.696561 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:57.696647 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:57.696699 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 19:16:57.697693 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:57.698447 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:57.700632 (+ 2185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:57.701776 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:57.702027 (+ 251us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:57.702121 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:57.702526 (+ 405us) server_negotiation.cc:299] Negotiation successful
0811 19:16:57.702690 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260811 19:16:57.703140 5352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:57.695110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.131:39339 (local address 127.3.61.129:54709)
0811 19:16:57.696110 (+ 1000us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:57.696132 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 19:16:57.696370 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:16:57.696880 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:16:57.696885 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:16:57.696931 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:16:57.697447 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:57.697466 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:57.698640 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:57.698644 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:57.700692 (+ 2048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:57.700717 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:57.702078 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:57.702086 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:57.702261 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:57.702290 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 19:16:57.702392 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 19:16:57.702555 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":850,"mutex_wait_us":226,"spinlock_wait_cycles":19200,"thread_start_us":612,"threads_started":1}
I20260811 19:16:57.703706 5353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:57.695194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.130:45741 (local address 127.3.61.129:37739)
0811 19:16:57.696196 (+ 1002us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:57.696210 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 19:16:57.696389 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:16:57.696883 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:16:57.696887 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:16:57.696934 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 19:16:57.697518 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:57.697528 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:57.698604 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:57.698609 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:57.700418 (+ 1809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:16:57.700437 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:16:57.702151 (+ 1714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:16:57.702162 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:16:57.702338 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:57.702368 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 19:16:57.702471 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 19:16:57.702645 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":880,"mutex_wait_us":124,"spinlock_wait_cycles":13696,"thread_start_us":1083,"threads_started":1}
I20260811 19:16:57.704653 5221 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48584db90ff642d59db61b2a91fcf968" candidate_uuid: "45ba5cdaa5f8446e947e5d13b56171bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44a8c55066964a32a0bae081e3774aaa" is_pre_election: true
I20260811 19:16:57.704766 5003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48584db90ff642d59db61b2a91fcf968" candidate_uuid: "45ba5cdaa5f8446e947e5d13b56171bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40c9c1ee2ba745dfb7225f98dc620e47" is_pre_election: true
I20260811 19:16:57.705317 5003 raft_consensus.cc:2454] T 48584db90ff642d59db61b2a91fcf968 P 40c9c1ee2ba745dfb7225f98dc620e47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ba5cdaa5f8446e947e5d13b56171bf in term 0.
I20260811 19:16:57.705320 5221 raft_consensus.cc:2454] T 48584db90ff642d59db61b2a91fcf968 P 44a8c55066964a32a0bae081e3774aaa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45ba5cdaa5f8446e947e5d13b56171bf in term 0.
I20260811 19:16:57.706266 4666 leader_election.cc:302] T 48584db90ff642d59db61b2a91fcf968 P 45ba5cdaa5f8446e947e5d13b56171bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44a8c55066964a32a0bae081e3774aaa, 45ba5cdaa5f8446e947e5d13b56171bf; no voters:
I20260811 19:16:57.706702 5349 raft_consensus.cc:2781] T 48584db90ff642d59db61b2a91fcf968 P 45ba5cdaa5f8446e947e5d13b56171bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 19:16:57.706856 5349 raft_consensus.cc:482] T 48584db90ff642d59db61b2a91fcf968 P 45ba5cdaa5f8446e947e5d13b56171bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 19:16:57.706971 5349 raft_consensus.cc:3037] T 48584db90ff642d59db61b2a91fcf968 P 45ba5cdaa5f8446e947e5d13b56171bf [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:16:57.709235 5349 raft_consensus.cc:504] T 48584db90ff642d59db61b2a91fcf968 P 45ba5cdaa5f8446e947e5d13b56171bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44a8c55066964a32a0bae081e3774aaa" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39339 } } peers { permanent_uuid: "45ba5cdaa5f8446e947e5d13b56171bf" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 43271 } } peers { permanent_uuid: "40c9c1ee2ba745dfb7225f98dc620e47" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45741 } }
I20260811 19:16:57.709990 5349 leader_election.cc:288] T 48584db90ff642d59db61b2a91fcf968 P 45ba5cdaa5f8446e947e5d13b56171bf [CANDIDATE]: Term 1 election: Requested vote from peers 44a8c55066964a32a0bae081e3774aaa (127.3.61.131:39339), 40c9c1ee2ba745dfb7225f98dc620e47 (127.3.61.130:45741)
I20260811 19:16:57.710604 5221 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48584db90ff642d59db61b2a91fcf968" candidate_uuid: "45ba5cdaa5f8446e947e5d13b56171bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44a8c55066964a32a0bae081e3774aaa"
I20260811 19:16:57.710608 5003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48584db90ff642d59db61b2a91fcf968" candidate_uuid: "45ba5cdaa5f8446e947e5d13b56171bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40c9c1ee2ba745dfb7225f98dc620e47"
I20260811 19:16:57.710829 5221 raft_consensus.cc:3037] T 48584db90ff642d59db61b2a91fcf968 P 44a8c55066964a32a0bae081e3774aaa [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:16:57.710816 5003 raft_consensus.cc:3037] T 48584db90ff642d59db61b2a91fcf968 P 40c9c1ee2ba745dfb7225f98dc620e47 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:16:57.713284 5221 raft_consensus.cc:2454] T 48584db90ff642d59db61b2a91fcf968 P 44a8c55066964a32a0bae081e3774aaa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ba5cdaa5f8446e947e5d13b56171bf in term 1.
I20260811 19:16:57.713366 5003 raft_consensus.cc:2454] T 48584db90ff642d59db61b2a91fcf968 P 40c9c1ee2ba745dfb7225f98dc620e47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45ba5cdaa5f8446e947e5d13b56171bf in term 1.
I20260811 19:16:57.713858 4666 leader_election.cc:302] T 48584db90ff642d59db61b2a91fcf968 P 45ba5cdaa5f8446e947e5d13b56171bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 40c9c1ee2ba745dfb7225f98dc620e47, 44a8c55066964a32a0bae081e3774aaa, 45ba5cdaa5f8446e947e5d13b56171bf; no voters:
I20260811 19:16:57.714296 5349 raft_consensus.cc:2781] T 48584db90ff642d59db61b2a91fcf968 P 45ba5cdaa5f8446e947e5d13b56171bf [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:16:57.714917 5349 raft_consensus.cc:686] T 48584db90ff642d59db61b2a91fcf968 P 45ba5cdaa5f8446e947e5d13b56171bf [term 1 LEADER]: Becoming Leader. State: Replica: 45ba5cdaa5f8446e947e5d13b56171bf, State: Running, Role: LEADER
I20260811 19:16:57.715512 5349 consensus_queue.cc:237] T 48584db90ff642d59db61b2a91fcf968 P 45ba5cdaa5f8446e947e5d13b56171bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44a8c55066964a32a0bae081e3774aaa" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39339 } } peers { permanent_uuid: "45ba5cdaa5f8446e947e5d13b56171bf" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 43271 } } peers { permanent_uuid: "40c9c1ee2ba745dfb7225f98dc620e47" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45741 } }
I20260811 19:16:57.721292 4529 catalog_manager.cc:5495] T 48584db90ff642d59db61b2a91fcf968 P 45ba5cdaa5f8446e947e5d13b56171bf reported cstate change: term changed from 0 to 1, leader changed from <none> to 45ba5cdaa5f8446e947e5d13b56171bf (127.3.61.129). New cstate: current_term: 1 leader_uuid: "45ba5cdaa5f8446e947e5d13b56171bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44a8c55066964a32a0bae081e3774aaa" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45ba5cdaa5f8446e947e5d13b56171bf" member_type: VOTER last_known_addr { host: "127.3.61.129" port: 43271 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40c9c1ee2ba745dfb7225f98dc620e47" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45741 } health_report { overall_health: UNKNOWN } } }
W20260811 19:16:57.754320 4865 tablet.cc:2369] T 48584db90ff642d59db61b2a91fcf968 P 45ba5cdaa5f8446e947e5d13b56171bf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:16:57.780558 5327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:57.773841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52624 (local address 127.3.61.129:43271)
0811 19:16:57.774040 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:57.774045 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:57.774069 (+ 24us) server_negotiation.cc:407] Connection header received
0811 19:16:57.774269 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:57.774274 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:57.774386 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:57.774547 (+ 161us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:16:57.774999 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:57.775836 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:57.777786 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:57.778278 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:57.778408 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:57.779064 (+ 656us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:16:57.779663 (+ 599us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:16:57.779790 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:57.780157 (+ 367us) server_negotiation.cc:299] Negotiation successful
0811 19:16:57.780275 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 19:16:57.791953 5221 raft_consensus.cc:1261] T 48584db90ff642d59db61b2a91fcf968 P 44a8c55066964a32a0bae081e3774aaa [term 1 FOLLOWER]: Refusing update from remote peer 45ba5cdaa5f8446e947e5d13b56171bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:16:57.791955 5003 raft_consensus.cc:1261] T 48584db90ff642d59db61b2a91fcf968 P 40c9c1ee2ba745dfb7225f98dc620e47 [term 1 FOLLOWER]: Refusing update from remote peer 45ba5cdaa5f8446e947e5d13b56171bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:16:57.793154 5354 consensus_queue.cc:1035] T 48584db90ff642d59db61b2a91fcf968 P 45ba5cdaa5f8446e947e5d13b56171bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "44a8c55066964a32a0bae081e3774aaa" member_type: VOTER last_known_addr { host: "127.3.61.131" port: 39339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:16:57.793706 5349 consensus_queue.cc:1035] T 48584db90ff642d59db61b2a91fcf968 P 45ba5cdaa5f8446e947e5d13b56171bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "40c9c1ee2ba745dfb7225f98dc620e47" member_type: VOTER last_known_addr { host: "127.3.61.130" port: 45741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:16:57.810470 5363 mvcc.cc:205] Tried to move back new op lower bound from 7317404949662203904 to 7317404949368025088. Current Snapshot: MvccSnapshot[applied={T|T < 7317404949662203904}]
I20260811 19:16:57.826669 4529 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45768:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:16:57.827080 4529 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45768:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 19:16:57.832407 4529 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 41c487474a9b46629ca620a804fb5ff8: Sending DeleteTablet for 3 replicas of tablet 48584db90ff642d59db61b2a91fcf968
I20260811 19:16:57.834007 5188 tablet_service.cc:1514] Processing DeleteTablet for tablet 48584db90ff642d59db61b2a91fcf968 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:16:57 UTC) from {username='slave'} at 127.0.0.1:39130
I20260811 19:16:57.834107 4969 tablet_service.cc:1514] Processing DeleteTablet for tablet 48584db90ff642d59db61b2a91fcf968 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:16:57 UTC) from {username='slave'} at 127.0.0.1:36972
I20260811 19:16:57.834544 4744 tablet_service.cc:1514] Processing DeleteTablet for tablet 48584db90ff642d59db61b2a91fcf968 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 19:16:57 UTC) from {username='slave'} at 127.0.0.1:52608
I20260811 19:16:57.836048 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 4640
I20260811 19:16:57.846529 5384 tablet_replica.cc:330] T 48584db90ff642d59db61b2a91fcf968 P 40c9c1ee2ba745dfb7225f98dc620e47: stopping tablet replica
I20260811 19:16:57.847152 5384 raft_consensus.cc:2229] T 48584db90ff642d59db61b2a91fcf968 P 40c9c1ee2ba745dfb7225f98dc620e47 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:16:57.847801 5384 raft_consensus.cc:2258] T 48584db90ff642d59db61b2a91fcf968 P 40c9c1ee2ba745dfb7225f98dc620e47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:16:57.849216 5385 tablet_replica.cc:330] T 48584db90ff642d59db61b2a91fcf968 P 45ba5cdaa5f8446e947e5d13b56171bf: stopping tablet replica
I20260811 19:16:57.849888 5385 raft_consensus.cc:2229] T 48584db90ff642d59db61b2a91fcf968 P 45ba5cdaa5f8446e947e5d13b56171bf [term 1 LEADER]: Raft consensus shutting down.
I20260811 19:16:57.850811 5385 raft_consensus.cc:2258] T 48584db90ff642d59db61b2a91fcf968 P 45ba5cdaa5f8446e947e5d13b56171bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:16:57.851250 5384 ts_tablet_manager.cc:1905] T 48584db90ff642d59db61b2a91fcf968 P 40c9c1ee2ba745dfb7225f98dc620e47: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:16:57.854089 5385 ts_tablet_manager.cc:1905] T 48584db90ff642d59db61b2a91fcf968 P 45ba5cdaa5f8446e947e5d13b56171bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:16:57.857468 5327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:57.837864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52640 (local address 127.3.61.129:43271)
0811 19:16:57.838206 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:57.838214 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:57.843122 (+ 4908us) server_negotiation.cc:407] Connection header received
0811 19:16:57.844494 (+ 1372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:57.844502 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:57.844884 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:57.845035 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:16:57.848240 (+ 3205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:57.849758 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:57.852274 (+ 2516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:57.852924 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:57.853051 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:57.854929 (+ 1878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:16:57.854958 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:16:57.854986 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:16:57.856184 (+ 1198us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:16:57.856265 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:16:57.856304 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:16:57.856413 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:57.856802 (+ 389us) server_negotiation.cc:299] Negotiation successful
0811 19:16:57.856955 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191}
I20260811 19:16:57.858678 4852 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:16:57.860915 5384 ts_tablet_manager.cc:1918] T 48584db90ff642d59db61b2a91fcf968 P 40c9c1ee2ba745dfb7225f98dc620e47: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:16:57.861238 5384 log.cc:1196] T 48584db90ff642d59db61b2a91fcf968 P 40c9c1ee2ba745dfb7225f98dc620e47: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/ts-1/wal/wals/48584db90ff642d59db61b2a91fcf968
I20260811 19:16:57.862253 5384 ts_tablet_manager.cc:1939] T 48584db90ff642d59db61b2a91fcf968 P 40c9c1ee2ba745dfb7225f98dc620e47: Deleting consensus metadata
I20260811 19:16:57.863502 5383 tablet_replica.cc:330] T 48584db90ff642d59db61b2a91fcf968 P 44a8c55066964a32a0bae081e3774aaa: stopping tablet replica
I20260811 19:16:57.864015 5383 raft_consensus.cc:2229] T 48584db90ff642d59db61b2a91fcf968 P 44a8c55066964a32a0bae081e3774aaa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 19:16:57.864598 5383 raft_consensus.cc:2258] T 48584db90ff642d59db61b2a91fcf968 P 44a8c55066964a32a0bae081e3774aaa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 19:16:57.864904 4502 catalog_manager.cc:4841] TS 40c9c1ee2ba745dfb7225f98dc620e47 (127.3.61.130:45741): tablet 48584db90ff642d59db61b2a91fcf968 (table test-table [id=dea961f2c33d46a89778704da0db0db1]) successfully deleted
I20260811 19:16:57.867718 5383 ts_tablet_manager.cc:1905] T 48584db90ff642d59db61b2a91fcf968 P 44a8c55066964a32a0bae081e3774aaa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 19:16:57.876031 5383 ts_tablet_manager.cc:1918] T 48584db90ff642d59db61b2a91fcf968 P 44a8c55066964a32a0bae081e3774aaa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 19:16:57.876260 5383 log.cc:1196] T 48584db90ff642d59db61b2a91fcf968 P 44a8c55066964a32a0bae081e3774aaa: Deleting WAL directory at /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786475454804547-3318-0/minicluster-data/ts-2/wal/wals/48584db90ff642d59db61b2a91fcf968
I20260811 19:16:57.876780 5383 ts_tablet_manager.cc:1939] T 48584db90ff642d59db61b2a91fcf968 P 44a8c55066964a32a0bae081e3774aaa: Deleting consensus metadata
I20260811 19:16:57.879093 4498 catalog_manager.cc:4841] TS 44a8c55066964a32a0bae081e3774aaa (127.3.61.131:39339): tablet 48584db90ff642d59db61b2a91fcf968 (table test-table [id=dea961f2c33d46a89778704da0db0db1]) successfully deleted
I20260811 19:16:57.974862 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 4640
W20260811 19:16:58.000432 4501 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.61.129:43271 (error 108)
I20260811 19:16:58.000559 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 4869
I20260811 19:16:58.002802 5330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:58.001875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:43271 (local address 127.0.0.1:52652)
0811 19:16:58.002265 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:58.002535 (+ 270us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.129:43271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137}
W20260811 19:16:58.003157 4501 catalog_manager.cc:4584] TS 45ba5cdaa5f8446e947e5d13b56171bf (127.3.61.129:43271): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 48584db90ff642d59db61b2a91fcf968: Network error: Client connection negotiation failed: client connection to 127.3.61.129:43271: connect: Connection refused (error 111)
I20260811 19:16:58.009618 5328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:58.001372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36976 (local address 127.3.61.130:45741)
0811 19:16:58.001581 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:58.001585 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:58.001605 (+ 20us) server_negotiation.cc:407] Connection header received
0811 19:16:58.001833 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:58.001839 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:58.001953 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:58.002077 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:16:58.003024 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:58.004238 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:58.005928 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:58.006484 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:58.006652 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:58.007968 (+ 1316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:16:58.007982 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:16:58.008001 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:16:58.008722 (+ 721us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:16:58.008786 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:16:58.008810 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:16:58.008928 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:58.009174 (+ 246us) server_negotiation.cc:299] Negotiation successful
0811 19:16:58.009305 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 19:16:58.010548 5080 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:16:58.056990 5330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:58.056446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:43271 (local address 127.0.0.1:52664)
0811 19:16:58.056694 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:58.056827 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.129:43271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260811 19:16:58.058126 5330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:58.057781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:43271 (local address 127.0.0.1:52674)
0811 19:16:58.057916 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:58.057989 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.129:43271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260811 19:16:58.106524 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 4869
I20260811 19:16:58.119613 5330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:58.118893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:43271 (local address 127.0.0.1:52686)
0811 19:16:58.119269 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:58.119397 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.129:43271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211}
I20260811 19:16:58.121089 5330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:58.120566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:43271 (local address 127.0.0.1:52698)
0811 19:16:58.120809 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:58.120912 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.129:43271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126}
I20260811 19:16:58.142864 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 5097
I20260811 19:16:58.150462 5325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:58.143984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39132 (local address 127.3.61.131:39339)
0811 19:16:58.144190 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 19:16:58.144197 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:16:58.144218 (+ 21us) server_negotiation.cc:407] Connection header received
0811 19:16:58.144363 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:16:58.144368 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:16:58.144477 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:16:58.144568 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:16:58.144990 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:58.145756 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:58.147030 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:16:58.147428 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:16:58.147542 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:16:58.148973 (+ 1431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:16:58.148982 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:16:58.149000 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:16:58.149619 (+ 619us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:16:58.149730 (+ 111us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:16:58.149753 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:16:58.149864 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 19:16:58.150001 (+ 137us) server_negotiation.cc:299] Negotiation successful
0811 19:16:58.150123 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 19:16:58.151454 5294 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:16:58.202441 5330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:58.201831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:43271 (local address 127.0.0.1:52710)
0811 19:16:58.202098 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:58.202219 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.129:43271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260811 19:16:58.204128 5330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:58.203336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:43271 (local address 127.0.0.1:52726)
0811 19:16:58.203762 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:58.203905 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.129:43271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243}
I20260811 19:16:58.252696 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 5097
I20260811 19:16:58.287802 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 4473
I20260811 19:16:58.289566 4589 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:16:58.408342 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 4473
I20260811 19:16:58.408491 5330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:16:58.407884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.129:43271 (local address 127.0.0.1:52736)
0811 19:16:58.408083 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0811 19:16:58.408191 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.129:43271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
2026-08-11T19:16:58Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (3398 ms)
[----------] 6 tests from EncryptionPolicyTest (25640 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786475454804547-3318-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:16:58 dist-test-slave-t7g5 krb5kdc[5400](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:16:58 dist-test-slave-t7g5 krb5kdc[5400](info): set up 2 sockets
Aug 11 19:16:58 dist-test-slave-t7g5 krb5kdc[5400](info): commencing operation
krb5kdc: starting...
W20260811 19:16:59.789530 3318 mini_kdc.cc:121] Time spent starting KDC: real 1.322s user 0.000s sys 0.066s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Aug 11 19:17:00 dist-test-slave-t7g5 krb5kdc[5400](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475820, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:17:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:17:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786475454804547-3318-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786475454804547-3318-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 19:17:00.191594 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786475454804547-3318-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786475454804547-3318-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786475454804547-3318-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35191
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786475454804547-3318-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35191
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:17:00.615722 5425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:17:00.616184 5425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:17:00.616346 5425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:17:00.624078 5425 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:17:00.624236 5425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:17:00.624320 5425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:17:00.624491 5425 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:17:00.624646 5425 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:17:00.637146 5425 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786475454804547-3318-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786475454804547-3318-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35191
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786475454804547-3318-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35191
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786475454804547-3318-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:17:00.638733 5425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:17:00.640209 5425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:17:00.655483 5434 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:17:00.655435 5435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:17:00.655349 5437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:17:00.659977 5425 server_base.cc:1034] running on GCE node
I20260811 19:17:00.661351 5425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:17:00.663748 5425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:17:00.665154 5425 hybrid_clock.cc:648] HybridClock initialized: now 1786475820665046 us; error 60 us; skew 500 ppm
I20260811 19:17:00.665903 5425 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:17:00 dist-test-slave-t7g5 krb5kdc[5400](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475820, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:17:00.670331 5425 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 19:17:00.672622 5425 webserver.cc:460] Webserver started at http://127.0.0.1:40785/ using document root <none> and password file <none>
I20260811 19:17:00.673502 5425 fs_manager.cc:359] Metadata directory not provided
I20260811 19:17:00.673626 5425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:17:00.674027 5425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:17:00.677392 5425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786475454804547-3318-0/minicluster-data/master-0/data/instance:
uuid: "dcc4dd03a7e940a7931be733313333f7"
format_stamp: "Formatted at 2026-08-11 19:17:00 on dist-test-slave-t7g5"
server_key: "436408ef8bc127a8e6b23e189cd3974b"
server_key_iv: "3860bb6042db33cccfc65258b4ca8450"
server_key_version: "encryptionkey@0"
I20260811 19:17:00.678323 5425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786475454804547-3318-0/minicluster-data/master-0/wal/instance:
uuid: "dcc4dd03a7e940a7931be733313333f7"
format_stamp: "Formatted at 2026-08-11 19:17:00 on dist-test-slave-t7g5"
server_key: "436408ef8bc127a8e6b23e189cd3974b"
server_key_iv: "3860bb6042db33cccfc65258b4ca8450"
server_key_version: "encryptionkey@0"
I20260811 19:17:00.684358 5425 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 19:17:00.689049 5451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:17:00.690923 5425 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260811 19:17:00.691154 5425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786475454804547-3318-0/minicluster-data/master-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786475454804547-3318-0/minicluster-data/master-0/wal
uuid: "dcc4dd03a7e940a7931be733313333f7"
format_stamp: "Formatted at 2026-08-11 19:17:00 on dist-test-slave-t7g5"
server_key: "436408ef8bc127a8e6b23e189cd3974b"
server_key_iv: "3860bb6042db33cccfc65258b4ca8450"
server_key_version: "encryptionkey@0"
I20260811 19:17:00.691396 5425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786475454804547-3318-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786475454804547-3318-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786475454804547-3318-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:17:00.721253 5425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:17:00.726459 5425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:17:00.726804 5425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:17:00.749029 5425 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35191
I20260811 19:17:00.749069 5541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35191 every 8 connection(s)
I20260811 19:17:00.750751 5425 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
I20260811 19:17:00.755565 5542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:17:00.760965 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 5425
I20260811 19:17:00.761250 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786475454804547-3318-0/minicluster-data/master-0/wal/instance
I20260811 19:17:00.762298 3318 external_mini_cluster.cc:1428] Setting key 694e22c5a1eb0d82cc981432b6f9bd61
I20260811 19:17:00.770635 5542 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcc4dd03a7e940a7931be733313333f7: Bootstrap starting.
I20260811 19:17:00.774998 5542 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dcc4dd03a7e940a7931be733313333f7: Neither blocks nor log segments found. Creating new log.
I20260811 19:17:00.776453 5542 log.cc:824] T 00000000000000000000000000000000 P dcc4dd03a7e940a7931be733313333f7: Log is configured to *not* fsync() on all Append() calls
I20260811 19:17:00.780882 5542 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcc4dd03a7e940a7931be733313333f7: No bootstrap required, opened a new log
Aug 11 19:17:00 dist-test-slave-t7g5 krb5kdc[5400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475820, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 19:17:00.790171 5542 raft_consensus.cc:348] T 00000000000000000000000000000000 P dcc4dd03a7e940a7931be733313333f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcc4dd03a7e940a7931be733313333f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35191 } }
I20260811 19:17:00.790691 5542 raft_consensus.cc:374] T 00000000000000000000000000000000 P dcc4dd03a7e940a7931be733313333f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:17:00.790808 5542 raft_consensus.cc:729] T 00000000000000000000000000000000 P dcc4dd03a7e940a7931be733313333f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcc4dd03a7e940a7931be733313333f7, State: Initialized, Role: FOLLOWER
I20260811 19:17:00.791595 5542 consensus_queue.cc:260] T 00000000000000000000000000000000 P dcc4dd03a7e940a7931be733313333f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcc4dd03a7e940a7931be733313333f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35191 } }
I20260811 19:17:00.791867 5542 raft_consensus.cc:388] T 00000000000000000000000000000000 P dcc4dd03a7e940a7931be733313333f7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:17:00.792019 5542 raft_consensus.cc:482] T 00000000000000000000000000000000 P dcc4dd03a7e940a7931be733313333f7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:17:00.792222 5542 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dcc4dd03a7e940a7931be733313333f7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:17:00.793326 5547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:00.764802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44869 (local address 127.0.0.1:35191)
0811 19:17:00.765806 (+ 1004us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:00.765819 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:00.765869 (+ 50us) server_negotiation.cc:407] Connection header received
0811 19:17:00.768063 (+ 2194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:00.768095 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:00.768614 (+ 519us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:00.769016 (+ 402us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:00.771664 (+ 2648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:00.773696 (+ 2032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:00.775832 (+ 2136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:00.776863 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:00.777127 (+ 264us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:00.782319 (+ 5192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:00.782330 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:00.782348 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:17:00.782396 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:00.786752 (+ 4356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:00.788118 (+ 1366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:00.788132 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:00.788144 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:00.788421 (+ 277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:00.789017 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:00.789021 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:00.789024 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:00.789953 (+ 929us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:00.790374 (+ 421us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:00.791009 (+ 635us) server_negotiation.cc:299] Negotiation successful
0811 19:17:00.791550 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":735,"thread_start_us":473,"threads_started":1}
I20260811 19:17:00.795692 5542 raft_consensus.cc:504] T 00000000000000000000000000000000 P dcc4dd03a7e940a7931be733313333f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcc4dd03a7e940a7931be733313333f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35191 } }
I20260811 19:17:00.796326 5542 leader_election.cc:302] T 00000000000000000000000000000000 P dcc4dd03a7e940a7931be733313333f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dcc4dd03a7e940a7931be733313333f7; no voters:
I20260811 19:17:00.797133 5542 leader_election.cc:288] T 00000000000000000000000000000000 P dcc4dd03a7e940a7931be733313333f7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:17:00.797292 5550 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dcc4dd03a7e940a7931be733313333f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:17:00.798880 5550 raft_consensus.cc:686] T 00000000000000000000000000000000 P dcc4dd03a7e940a7931be733313333f7 [term 1 LEADER]: Becoming Leader. State: Replica: dcc4dd03a7e940a7931be733313333f7, State: Running, Role: LEADER
I20260811 19:17:00.799173 3318 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:17:00.799559 5542 sys_catalog.cc:564] T 00000000000000000000000000000000 P dcc4dd03a7e940a7931be733313333f7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:17:00.799539 5550 consensus_queue.cc:237] T 00000000000000000000000000000000 P dcc4dd03a7e940a7931be733313333f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcc4dd03a7e940a7931be733313333f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35191 } }
I20260811 19:17:00.802564 5553 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcc4dd03a7e940a7931be733313333f7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dcc4dd03a7e940a7931be733313333f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcc4dd03a7e940a7931be733313333f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35191 } } }
I20260811 19:17:00.802989 5553 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcc4dd03a7e940a7931be733313333f7 [sys.catalog]: This master's current role is: LEADER
I20260811 19:17:00.803416 5560 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcc4dd03a7e940a7931be733313333f7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dcc4dd03a7e940a7931be733313333f7. Latest consensus state: current_term: 1 leader_uuid: "dcc4dd03a7e940a7931be733313333f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcc4dd03a7e940a7931be733313333f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35191 } } }
I20260811 19:17:00.803704 5560 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcc4dd03a7e940a7931be733313333f7 [sys.catalog]: This master's current role is: LEADER
I20260811 19:17:00.807103 5578 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:17:00.816232 5578 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:17:00.830015 5547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:00.810435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58395 (local address 127.0.0.1:35191)
0811 19:17:00.810715 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:00.810721 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:00.810746 (+ 25us) server_negotiation.cc:407] Connection header received
0811 19:17:00.810832 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:00.810842 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:00.810935 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:00.811065 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:00.814113 (+ 3048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:00.815420 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:00.817834 (+ 2414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:00.818580 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:00.818696 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:00.822645 (+ 3949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:00.822651 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:00.822655 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:17:00.822696 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:00.826548 (+ 3852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:00.827686 (+ 1138us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:00.827696 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:00.827699 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:00.827839 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:00.828319 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:00.828324 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:00.828328 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:00.828727 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:00.829022 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:00.829457 (+ 435us) server_negotiation.cc:299] Negotiation successful
0811 19:17:00.829690 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128}
I20260811 19:17:00.834494 5614 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 19:17:00.837265 5578 catalog_manager.cc:1347] Generated new cluster ID: cbe0eaf99d2448ec9e385e72e76e97de
I20260811 19:17:00.837450 5578 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:17:00.856943 5578 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:17:00.859333 5578 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:17:00.873781 5578 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dcc4dd03a7e940a7931be733313333f7: Generated new TSK 0
I20260811 19:17:00.874812 5578 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:17:00.889025 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 5425
I20260811 19:17:00.891074 5536 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:17:00.992944 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 5425
W20260811 19:17:01.021517 5397 connection.cc:535] client connection to 127.0.0.1:35191 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35191): Connection reset by peer (error 104)
2026-08-11T19:17:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2575 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786475454804547-3318-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:17:01 dist-test-slave-t7g5 krb5kdc[5618](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:17:01 dist-test-slave-t7g5 krb5kdc[5618](info): set up 2 sockets
Aug 11 19:17:01 dist-test-slave-t7g5 krb5kdc[5618](info): commencing operation
krb5kdc: starting...
W20260811 19:17:02.355722 3318 mini_kdc.cc:121] Time spent starting KDC: real 1.313s user 0.000s sys 0.067s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Aug 11 19:17:02 dist-test-slave-t7g5 krb5kdc[5618](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475822, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:17:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:17:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786475454804547-3318-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786475454804547-3318-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 19:17:02.745222 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786475454804547-3318-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786475454804547-3318-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786475454804547-3318-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34643
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786475454804547-3318-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34643
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:17:03.159988 5645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:17:03.160357 5645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:17:03.160496 5645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:17:03.169768 5645 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:17:03.169966 5645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:17:03.170063 5645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:17:03.170151 5645 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:17:03.170213 5645 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:17:03.184296 5645 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786475454804547-3318-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786475454804547-3318-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34643
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786475454804547-3318-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34643
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786475454804547-3318-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:17:03.186383 5645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:17:03.188264 5645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:17:03.203464 5658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:17:03.203423 5663 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:17:03.203486 5657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:17:03.208244 5645 server_base.cc:1034] running on GCE node
I20260811 19:17:03.209306 5645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:17:03.210994 5645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:17:03.212262 5645 hybrid_clock.cc:648] HybridClock initialized: now 1786475823212209 us; error 42 us; skew 500 ppm
I20260811 19:17:03.212664 5645 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:17:03 dist-test-slave-t7g5 krb5kdc[5618](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475823, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:17:03.220674 5645 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 19:17:03.222760 5645 webserver.cc:460] Webserver started at http://127.0.0.1:46057/ using document root <none> and password file <none>
I20260811 19:17:03.224020 5645 fs_manager.cc:359] Metadata directory not provided
I20260811 19:17:03.224213 5645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:17:03.224740 5645 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:17:03.229489 5645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786475454804547-3318-0/minicluster-data/master-0/data/instance:
uuid: "6609ef09027f463ca93fd080aaac4f13"
format_stamp: "Formatted at 2026-08-11 19:17:03 on dist-test-slave-t7g5"
server_key: "137418d6a4c8f9087931604bc081f7d6"
server_key_iv: "d10953b23a14615d7f4311ceac90622f"
server_key_version: "encryptionkey@0"
I20260811 19:17:03.230602 5645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786475454804547-3318-0/minicluster-data/master-0/wal/instance:
uuid: "6609ef09027f463ca93fd080aaac4f13"
format_stamp: "Formatted at 2026-08-11 19:17:03 on dist-test-slave-t7g5"
server_key: "137418d6a4c8f9087931604bc081f7d6"
server_key_iv: "d10953b23a14615d7f4311ceac90622f"
server_key_version: "encryptionkey@0"
I20260811 19:17:03.238199 5645 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260811 19:17:03.244019 5681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:17:03.246937 5645 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.006s sys 0.000s
I20260811 19:17:03.247255 5645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786475454804547-3318-0/minicluster-data/master-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786475454804547-3318-0/minicluster-data/master-0/wal
uuid: "6609ef09027f463ca93fd080aaac4f13"
format_stamp: "Formatted at 2026-08-11 19:17:03 on dist-test-slave-t7g5"
server_key: "137418d6a4c8f9087931604bc081f7d6"
server_key_iv: "d10953b23a14615d7f4311ceac90622f"
server_key_version: "encryptionkey@0"
I20260811 19:17:03.247552 5645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786475454804547-3318-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786475454804547-3318-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786475454804547-3318-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:17:03.270500 5645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:17:03.275908 5645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:17:03.276242 5645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:17:03.298558 5645 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34643
I20260811 19:17:03.298575 5780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34643 every 8 connection(s)
I20260811 19:17:03.300158 5645 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
I20260811 19:17:03.305761 5781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:17:03.308770 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 5645
I20260811 19:17:03.308990 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786475454804547-3318-0/minicluster-data/master-0/wal/instance
I20260811 19:17:03.309720 3318 external_mini_cluster.cc:1428] Setting key 395e32fc8ee2d322531b4a61eaabddfc
I20260811 19:17:03.324779 5781 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6609ef09027f463ca93fd080aaac4f13: Bootstrap starting.
I20260811 19:17:03.330355 5781 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6609ef09027f463ca93fd080aaac4f13: Neither blocks nor log segments found. Creating new log.
I20260811 19:17:03.330950 5785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:03.312212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40233 (local address 127.0.0.1:34643)
0811 19:17:03.313429 (+ 1217us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:03.313452 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:03.313532 (+ 80us) server_negotiation.cc:407] Connection header received
0811 19:17:03.316320 (+ 2788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:03.316361 (+ 41us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:03.317100 (+ 739us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:03.317764 (+ 664us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:03.318497 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:03.321143 (+ 2646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:03.323217 (+ 2074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:03.324596 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:03.324842 (+ 246us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:03.326738 (+ 1896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:03.326751 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:03.326777 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:17:03.327763 (+ 986us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:03.327915 (+ 152us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:17:03.327959 (+ 44us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:03.328186 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:03.328717 (+ 531us) server_negotiation.cc:299] Negotiation successful
0811 19:17:03.328984 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":837,"thread_start_us":523,"threads_started":1}
I20260811 19:17:03.331935 5781 log.cc:824] T 00000000000000000000000000000000 P 6609ef09027f463ca93fd080aaac4f13: Log is configured to *not* fsync() on all Append() calls
I20260811 19:17:03.335871 5781 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6609ef09027f463ca93fd080aaac4f13: No bootstrap required, opened a new log
I20260811 19:17:03.336792 3318 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:17:03.345256 5781 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6609ef09027f463ca93fd080aaac4f13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6609ef09027f463ca93fd080aaac4f13" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34643 } }
I20260811 19:17:03.345894 5781 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6609ef09027f463ca93fd080aaac4f13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:17:03.346040 5781 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6609ef09027f463ca93fd080aaac4f13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6609ef09027f463ca93fd080aaac4f13, State: Initialized, Role: FOLLOWER
I20260811 19:17:03.346866 5781 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6609ef09027f463ca93fd080aaac4f13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6609ef09027f463ca93fd080aaac4f13" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34643 } }
I20260811 19:17:03.347213 5781 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6609ef09027f463ca93fd080aaac4f13 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:17:03.347383 5781 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6609ef09027f463ca93fd080aaac4f13 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:17:03.347568 5781 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6609ef09027f463ca93fd080aaac4f13 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:17:03.351728 5781 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6609ef09027f463ca93fd080aaac4f13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6609ef09027f463ca93fd080aaac4f13" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34643 } }
I20260811 19:17:03.352121 5785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:03.342879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35001 (local address 127.0.0.1:34643)
0811 19:17:03.343169 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:03.343177 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:03.343539 (+ 362us) server_negotiation.cc:407] Connection header received
0811 19:17:03.343813 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:03.343830 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:03.343928 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:03.344075 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:03.344741 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:03.345774 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:03.347991 (+ 2217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:03.348665 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:03.348818 (+ 153us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:03.350990 (+ 2172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:03.350997 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:03.351001 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:17:03.351029 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:03.351123 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:17:03.351146 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:03.351263 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:03.351630 (+ 367us) server_negotiation.cc:299] Negotiation successful
0811 19:17:03.351728 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":126}
I20260811 19:17:03.352453 5781 leader_election.cc:302] T 00000000000000000000000000000000 P 6609ef09027f463ca93fd080aaac4f13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6609ef09027f463ca93fd080aaac4f13; no voters:
I20260811 19:17:03.353226 5781 leader_election.cc:288] T 00000000000000000000000000000000 P 6609ef09027f463ca93fd080aaac4f13 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:17:03.353325 5812 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6609ef09027f463ca93fd080aaac4f13 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:17:03.355834 5781 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6609ef09027f463ca93fd080aaac4f13 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:17:03.360165 5812 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6609ef09027f463ca93fd080aaac4f13 [term 1 LEADER]: Becoming Leader. State: Replica: 6609ef09027f463ca93fd080aaac4f13, State: Running, Role: LEADER
I20260811 19:17:03.360971 5812 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6609ef09027f463ca93fd080aaac4f13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6609ef09027f463ca93fd080aaac4f13" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34643 } }
I20260811 19:17:03.364379 5813 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6609ef09027f463ca93fd080aaac4f13 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6609ef09027f463ca93fd080aaac4f13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6609ef09027f463ca93fd080aaac4f13" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34643 } } }
I20260811 19:17:03.364810 5813 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6609ef09027f463ca93fd080aaac4f13 [sys.catalog]: This master's current role is: LEADER
I20260811 19:17:03.365042 5814 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6609ef09027f463ca93fd080aaac4f13 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6609ef09027f463ca93fd080aaac4f13. Latest consensus state: current_term: 1 leader_uuid: "6609ef09027f463ca93fd080aaac4f13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6609ef09027f463ca93fd080aaac4f13" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34643 } } }
I20260811 19:17:03.365428 5814 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6609ef09027f463ca93fd080aaac4f13 [sys.catalog]: This master's current role is: LEADER
W20260811 19:17:03.383630 5834 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6609ef09027f463ca93fd080aaac4f13: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 19:17:03.383806 5834 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 19:17:03.383944 5836 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:17:03.385895 5836 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:17:03.396016 5836 catalog_manager.cc:1347] Generated new cluster ID: 4536b6a163274b92acf78b0450a58271
I20260811 19:17:03.396169 5836 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:17:03.417826 5836 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:17:03.419968 5836 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:17:03.447731 5836 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6609ef09027f463ca93fd080aaac4f13: Generated new TSK 0
I20260811 19:17:03.448725 5836 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:17:03.464179 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 5645
W20260811 19:17:03.465943 5774 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:40233
W20260811 19:17:03.467082 3318 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 5645: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 19:17:03.467320 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 5645
W20260811 19:17:03.511757 5616 connection.cc:535] client connection to 127.0.0.1:34643 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34643): Connection reset by peer (error 104)
2026-08-11T19:17:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2489 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786475454804547-3318-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:17:03 dist-test-slave-t7g5 krb5kdc[5844](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:17:03 dist-test-slave-t7g5 krb5kdc[5844](info): set up 2 sockets
Aug 11 19:17:03 dist-test-slave-t7g5 krb5kdc[5844](info): commencing operation
krb5kdc: starting...
W20260811 19:17:04.836498 3318 mini_kdc.cc:121] Time spent starting KDC: real 1.304s user 0.002s sys 0.065s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Aug 11 19:17:05 dist-test-slave-t7g5 krb5kdc[5844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475825, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:17:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:17:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786475454804547-3318-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786475454804547-3318-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 19:17:05.234308 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786475454804547-3318-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786475454804547-3318-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786475454804547-3318-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43277
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786475454804547-3318-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43277
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:17:05.645627 5873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:17:05.646031 5873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:17:05.646133 5873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:17:05.655712 5873 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:17:05.655865 5873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:17:05.655915 5873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:17:05.655978 5873 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:17:05.656024 5873 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:17:05.669325 5873 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786475454804547-3318-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786475454804547-3318-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43277
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786475454804547-3318-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43277
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786475454804547-3318-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:17:05.671216 5873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:17:05.672698 5873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:17:05.685737 5880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:17:05.685593 5888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:17:05.685627 5884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:17:05.686822 5873 server_base.cc:1034] running on GCE node
I20260811 19:17:05.688108 5873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:17:05.689898 5873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:17:05.691160 5873 hybrid_clock.cc:648] HybridClock initialized: now 1786475825691106 us; error 39 us; skew 500 ppm
I20260811 19:17:05.691563 5873 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:17:05 dist-test-slave-t7g5 krb5kdc[5844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475825, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:17:05.695056 5873 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 19:17:05.696923 5873 webserver.cc:460] Webserver started at http://127.0.0.1:46821/ using document root <none> and password file <none>
I20260811 19:17:05.697912 5873 fs_manager.cc:359] Metadata directory not provided
I20260811 19:17:05.698046 5873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:17:05.698346 5873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:17:05.702162 5873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786475454804547-3318-0/minicluster-data/master-0/data/instance:
uuid: "3af8e922f88b49fe849d44cab98b58ca"
format_stamp: "Formatted at 2026-08-11 19:17:05 on dist-test-slave-t7g5"
server_key: "bcffd59d484d9f05e299e03b1744fd13"
server_key_iv: "f60c97c5771196701cdbc412e2c066c4"
server_key_version: "encryptionkey@0"
I20260811 19:17:05.703269 5873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786475454804547-3318-0/minicluster-data/master-0/wal/instance:
uuid: "3af8e922f88b49fe849d44cab98b58ca"
format_stamp: "Formatted at 2026-08-11 19:17:05 on dist-test-slave-t7g5"
server_key: "bcffd59d484d9f05e299e03b1744fd13"
server_key_iv: "f60c97c5771196701cdbc412e2c066c4"
server_key_version: "encryptionkey@0"
I20260811 19:17:05.710116 5873 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260811 19:17:05.714437 5901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:17:05.716061 5873 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 19:17:05.716238 5873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786475454804547-3318-0/minicluster-data/master-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786475454804547-3318-0/minicluster-data/master-0/wal
uuid: "3af8e922f88b49fe849d44cab98b58ca"
format_stamp: "Formatted at 2026-08-11 19:17:05 on dist-test-slave-t7g5"
server_key: "bcffd59d484d9f05e299e03b1744fd13"
server_key_iv: "f60c97c5771196701cdbc412e2c066c4"
server_key_version: "encryptionkey@0"
I20260811 19:17:05.716456 5873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786475454804547-3318-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786475454804547-3318-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786475454804547-3318-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:17:05.723222 5873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:17:05.728741 5873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:17:05.729044 5873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:17:05.750965 5873 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43277
I20260811 19:17:05.751029 6011 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43277 every 8 connection(s)
I20260811 19:17:05.752564 5873 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
I20260811 19:17:05.755210 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 5873
I20260811 19:17:05.755406 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786475454804547-3318-0/minicluster-data/master-0/wal/instance
I20260811 19:17:05.755869 3318 external_mini_cluster.cc:1428] Setting key 96d5ffb76267b52fc8b3ca113d6ed739
I20260811 19:17:05.756465 6012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:17:05.767287 6017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:05.757871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54421 (local address 127.0.0.1:43277)
0811 19:17:05.758765 (+ 894us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:05.758777 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:05.758821 (+ 44us) server_negotiation.cc:407] Connection header received
0811 19:17:05.761939 (+ 3118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:05.762024 (+ 85us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:05.762658 (+ 634us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:05.763342 (+ 684us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:05.764990 (+ 1648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:05.764998 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:05.765015 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:17:05.765033 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:05.765105 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:17:05.765127 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:05.765395 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:05.765565 (+ 170us) server_negotiation.cc:299] Negotiation successful
0811 19:17:05.765807 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":659,"thread_start_us":381,"threads_started":1}
I20260811 19:17:05.771435 6012 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3af8e922f88b49fe849d44cab98b58ca: Bootstrap starting.
I20260811 19:17:05.772078 3318 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:17:05.774897 6012 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3af8e922f88b49fe849d44cab98b58ca: Neither blocks nor log segments found. Creating new log.
I20260811 19:17:05.776083 6012 log.cc:824] T 00000000000000000000000000000000 P 3af8e922f88b49fe849d44cab98b58ca: Log is configured to *not* fsync() on all Append() calls
I20260811 19:17:05.779667 6017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:05.777013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52087 (local address 127.0.0.1:43277)
0811 19:17:05.777238 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:05.777246 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:05.777545 (+ 299us) server_negotiation.cc:407] Connection header received
0811 19:17:05.777799 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:05.777803 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:05.777858 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:05.777963 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:05.778989 (+ 1026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:05.778993 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:05.778997 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:17:05.779013 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:05.779055 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:17:05.779071 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:05.779164 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:05.779305 (+ 141us) server_negotiation.cc:299] Negotiation successful
0811 19:17:05.779362 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 19:17:05.779845 6012 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3af8e922f88b49fe849d44cab98b58ca: No bootstrap required, opened a new log
I20260811 19:17:05.787730 6012 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3af8e922f88b49fe849d44cab98b58ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3af8e922f88b49fe849d44cab98b58ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43277 } }
I20260811 19:17:05.788166 6012 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3af8e922f88b49fe849d44cab98b58ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:17:05.788268 6012 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3af8e922f88b49fe849d44cab98b58ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3af8e922f88b49fe849d44cab98b58ca, State: Initialized, Role: FOLLOWER
I20260811 19:17:05.788928 6012 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3af8e922f88b49fe849d44cab98b58ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3af8e922f88b49fe849d44cab98b58ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43277 } }
I20260811 19:17:05.789155 6012 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3af8e922f88b49fe849d44cab98b58ca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:17:05.789300 6012 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3af8e922f88b49fe849d44cab98b58ca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:17:05.789463 6012 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3af8e922f88b49fe849d44cab98b58ca [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:17:05.791998 6012 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3af8e922f88b49fe849d44cab98b58ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3af8e922f88b49fe849d44cab98b58ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43277 } }
I20260811 19:17:05.792618 6012 leader_election.cc:302] T 00000000000000000000000000000000 P 3af8e922f88b49fe849d44cab98b58ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3af8e922f88b49fe849d44cab98b58ca; no voters:
I20260811 19:17:05.793243 6012 leader_election.cc:288] T 00000000000000000000000000000000 P 3af8e922f88b49fe849d44cab98b58ca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:17:05.793399 6027 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3af8e922f88b49fe849d44cab98b58ca [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:17:05.794106 6027 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3af8e922f88b49fe849d44cab98b58ca [term 1 LEADER]: Becoming Leader. State: Replica: 3af8e922f88b49fe849d44cab98b58ca, State: Running, Role: LEADER
I20260811 19:17:05.794665 6027 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3af8e922f88b49fe849d44cab98b58ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3af8e922f88b49fe849d44cab98b58ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43277 } }
I20260811 19:17:05.795524 6012 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3af8e922f88b49fe849d44cab98b58ca [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:17:05.798496 6032 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3af8e922f88b49fe849d44cab98b58ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3af8e922f88b49fe849d44cab98b58ca. Latest consensus state: current_term: 1 leader_uuid: "3af8e922f88b49fe849d44cab98b58ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3af8e922f88b49fe849d44cab98b58ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43277 } } }
I20260811 19:17:05.798949 6032 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3af8e922f88b49fe849d44cab98b58ca [sys.catalog]: This master's current role is: LEADER
I20260811 19:17:05.799367 6030 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3af8e922f88b49fe849d44cab98b58ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3af8e922f88b49fe849d44cab98b58ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3af8e922f88b49fe849d44cab98b58ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43277 } } }
I20260811 19:17:05.799656 6030 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3af8e922f88b49fe849d44cab98b58ca [sys.catalog]: This master's current role is: LEADER
I20260811 19:17:05.806833 6045 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:17:05.812891 6045 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:17:05.823846 6045 catalog_manager.cc:1347] Generated new cluster ID: b37413aad3684c029c67a6d4e76f841d
I20260811 19:17:05.823987 6045 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:17:05.838472 6045 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:17:05.838999 6045 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:17:05.849193 6045 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3af8e922f88b49fe849d44cab98b58ca: Generated new TSK 0
I20260811 19:17:05.850458 6045 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:17:05.856952 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 5873
W20260811 19:17:05.858369 6006 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:54421
W20260811 19:17:05.859256 3318 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 5873: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 19:17:05.859565 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 5873
2026-08-11T19:17:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2383 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786475454804547-3318-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:17:06 dist-test-slave-t7g5 krb5kdc[6068](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:17:06 dist-test-slave-t7g5 krb5kdc[6068](info): set up 2 sockets
Aug 11 19:17:06 dist-test-slave-t7g5 krb5kdc[6068](info): commencing operation
krb5kdc: starting...
W20260811 19:17:07.223975 3318 mini_kdc.cc:121] Time spent starting KDC: real 1.307s user 0.003s sys 0.059s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Aug 11 19:17:07 dist-test-slave-t7g5 krb5kdc[6068](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475827, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:17:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:17:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786475454804547-3318-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786475454804547-3318-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 19:17:07.618305 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786475454804547-3318-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786475454804547-3318-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786475454804547-3318-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41771
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786475454804547-3318-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41771
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:17:08.016482 6095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:17:08.016852 6095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:17:08.016999 6095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:17:08.024842 6095 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:17:08.025028 6095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:17:08.025118 6095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:17:08.025207 6095 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:17:08.025277 6095 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:17:08.036710 6095 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786475454804547-3318-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786475454804547-3318-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41771
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786475454804547-3318-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41771
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786475454804547-3318-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:17:08.038697 6095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:17:08.040767 6095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:17:08.053848 6109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:17:08.053889 6111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:17:08.053941 6113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:17:08.054574 6095 server_base.cc:1034] running on GCE node
I20260811 19:17:08.055522 6095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:17:08.057245 6095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:17:08.058496 6095 hybrid_clock.cc:648] HybridClock initialized: now 1786475828058439 us; error 43 us; skew 500 ppm
I20260811 19:17:08.058928 6095 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:17:08 dist-test-slave-t7g5 krb5kdc[6068](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475828, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:17:08.063274 6095 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 19:17:08.065428 6095 webserver.cc:460] Webserver started at http://127.0.0.1:43563/ using document root <none> and password file <none>
I20260811 19:17:08.066327 6095 fs_manager.cc:359] Metadata directory not provided
I20260811 19:17:08.066474 6095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:17:08.067022 6095 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:17:08.071246 6095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786475454804547-3318-0/minicluster-data/master-0/data/instance:
uuid: "e29aaf9f688b4c0ebedacdeb0b245709"
format_stamp: "Formatted at 2026-08-11 19:17:08 on dist-test-slave-t7g5"
server_key: "71b7062cbe2fb16fa089f48421bc90bf"
server_key_iv: "c87dc33791b02c70b7a66527bf5a7b9c"
server_key_version: "encryptionkey@0"
I20260811 19:17:08.072168 6095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786475454804547-3318-0/minicluster-data/master-0/wal/instance:
uuid: "e29aaf9f688b4c0ebedacdeb0b245709"
format_stamp: "Formatted at 2026-08-11 19:17:08 on dist-test-slave-t7g5"
server_key: "71b7062cbe2fb16fa089f48421bc90bf"
server_key_iv: "c87dc33791b02c70b7a66527bf5a7b9c"
server_key_version: "encryptionkey@0"
I20260811 19:17:08.078797 6095 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260811 19:17:08.083766 6125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:17:08.085976 6095 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.001s
I20260811 19:17:08.086333 6095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786475454804547-3318-0/minicluster-data/master-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786475454804547-3318-0/minicluster-data/master-0/wal
uuid: "e29aaf9f688b4c0ebedacdeb0b245709"
format_stamp: "Formatted at 2026-08-11 19:17:08 on dist-test-slave-t7g5"
server_key: "71b7062cbe2fb16fa089f48421bc90bf"
server_key_iv: "c87dc33791b02c70b7a66527bf5a7b9c"
server_key_version: "encryptionkey@0"
I20260811 19:17:08.086668 6095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786475454804547-3318-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786475454804547-3318-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786475454804547-3318-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:17:08.111337 6095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:17:08.116544 6095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:17:08.116845 6095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:17:08.139433 6095 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41771
I20260811 19:17:08.139562 6235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41771 every 8 connection(s)
I20260811 19:17:08.140938 6095 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
I20260811 19:17:08.141404 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 6095
I20260811 19:17:08.141584 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786475454804547-3318-0/minicluster-data/master-0/wal/instance
I20260811 19:17:08.142210 3318 external_mini_cluster.cc:1428] Setting key 5b9d2c0694059b458aa3deae0b96ba95
I20260811 19:17:08.146257 6238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 19:17:08 dist-test-slave-t7g5 krb5kdc[6068](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475827, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 19:17:08.161684 6238 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e29aaf9f688b4c0ebedacdeb0b245709: Bootstrap starting.
I20260811 19:17:08.166249 6238 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e29aaf9f688b4c0ebedacdeb0b245709: Neither blocks nor log segments found. Creating new log.
I20260811 19:17:08.167670 6238 log.cc:824] T 00000000000000000000000000000000 P e29aaf9f688b4c0ebedacdeb0b245709: Log is configured to *not* fsync() on all Append() calls
I20260811 19:17:08.170127 6244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:08.144358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58929 (local address 127.0.0.1:41771)
0811 19:17:08.145186 (+ 828us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:08.145198 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:08.145243 (+ 45us) server_negotiation.cc:407] Connection header received
0811 19:17:08.147079 (+ 1836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:08.147104 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:08.147527 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:08.147896 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:08.149454 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:08.151234 (+ 1780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:08.152898 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:08.154082 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:08.154351 (+ 269us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:08.159682 (+ 5331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:08.159693 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:08.159713 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:17:08.159765 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:08.163535 (+ 3770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:08.164764 (+ 1229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:08.164776 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:08.164787 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:08.165098 (+ 311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:08.165760 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:08.165767 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:08.165769 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:08.166563 (+ 794us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:08.166886 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:08.167404 (+ 518us) server_negotiation.cc:299] Negotiation successful
0811 19:17:08.168175 (+ 771us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":365,"threads_started":1}
I20260811 19:17:08.171743 6238 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e29aaf9f688b4c0ebedacdeb0b245709: No bootstrap required, opened a new log
I20260811 19:17:08.174814 3318 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:17:08.182236 6238 raft_consensus.cc:348] T 00000000000000000000000000000000 P e29aaf9f688b4c0ebedacdeb0b245709 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e29aaf9f688b4c0ebedacdeb0b245709" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41771 } }
I20260811 19:17:08.182655 6238 raft_consensus.cc:374] T 00000000000000000000000000000000 P e29aaf9f688b4c0ebedacdeb0b245709 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:17:08.182761 6238 raft_consensus.cc:729] T 00000000000000000000000000000000 P e29aaf9f688b4c0ebedacdeb0b245709 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e29aaf9f688b4c0ebedacdeb0b245709, State: Initialized, Role: FOLLOWER
I20260811 19:17:08.183459 6238 consensus_queue.cc:260] T 00000000000000000000000000000000 P e29aaf9f688b4c0ebedacdeb0b245709 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e29aaf9f688b4c0ebedacdeb0b245709" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41771 } }
I20260811 19:17:08.183771 6238 raft_consensus.cc:388] T 00000000000000000000000000000000 P e29aaf9f688b4c0ebedacdeb0b245709 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:17:08.183928 6238 raft_consensus.cc:482] T 00000000000000000000000000000000 P e29aaf9f688b4c0ebedacdeb0b245709 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:17:08.184116 6238 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e29aaf9f688b4c0ebedacdeb0b245709 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:17:08.188259 6238 raft_consensus.cc:504] T 00000000000000000000000000000000 P e29aaf9f688b4c0ebedacdeb0b245709 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e29aaf9f688b4c0ebedacdeb0b245709" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41771 } }
I20260811 19:17:08.188936 6238 leader_election.cc:302] T 00000000000000000000000000000000 P e29aaf9f688b4c0ebedacdeb0b245709 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e29aaf9f688b4c0ebedacdeb0b245709; no voters:
I20260811 19:17:08.189756 6238 leader_election.cc:288] T 00000000000000000000000000000000 P e29aaf9f688b4c0ebedacdeb0b245709 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:17:08.190049 6259 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e29aaf9f688b4c0ebedacdeb0b245709 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:17:08.194443 6259 raft_consensus.cc:686] T 00000000000000000000000000000000 P e29aaf9f688b4c0ebedacdeb0b245709 [term 1 LEADER]: Becoming Leader. State: Replica: e29aaf9f688b4c0ebedacdeb0b245709, State: Running, Role: LEADER
I20260811 19:17:08.195283 6259 consensus_queue.cc:237] T 00000000000000000000000000000000 P e29aaf9f688b4c0ebedacdeb0b245709 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e29aaf9f688b4c0ebedacdeb0b245709" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41771 } }
I20260811 19:17:08.195827 6238 sys_catalog.cc:564] T 00000000000000000000000000000000 P e29aaf9f688b4c0ebedacdeb0b245709 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:17:08.199250 6260 sys_catalog.cc:455] T 00000000000000000000000000000000 P e29aaf9f688b4c0ebedacdeb0b245709 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e29aaf9f688b4c0ebedacdeb0b245709" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e29aaf9f688b4c0ebedacdeb0b245709" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41771 } } }
I20260811 19:17:08.199806 6260 sys_catalog.cc:458] T 00000000000000000000000000000000 P e29aaf9f688b4c0ebedacdeb0b245709 [sys.catalog]: This master's current role is: LEADER
I20260811 19:17:08.200385 6261 sys_catalog.cc:455] T 00000000000000000000000000000000 P e29aaf9f688b4c0ebedacdeb0b245709 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e29aaf9f688b4c0ebedacdeb0b245709. Latest consensus state: current_term: 1 leader_uuid: "e29aaf9f688b4c0ebedacdeb0b245709" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e29aaf9f688b4c0ebedacdeb0b245709" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41771 } } }
I20260811 19:17:08.200708 6261 sys_catalog.cc:458] T 00000000000000000000000000000000 P e29aaf9f688b4c0ebedacdeb0b245709 [sys.catalog]: This master's current role is: LEADER
I20260811 19:17:08.201486 6244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:08.180441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47737 (local address 127.0.0.1:41771)
0811 19:17:08.180659 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:08.180666 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:08.181051 (+ 385us) server_negotiation.cc:407] Connection header received
0811 19:17:08.181205 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:08.181211 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:08.181293 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:08.181422 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:08.183241 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:08.184318 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:08.185965 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:08.186984 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:08.187310 (+ 326us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:08.190170 (+ 2860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:08.190177 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:08.190182 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:17:08.190228 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:08.197155 (+ 6927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:08.198377 (+ 1222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:08.198383 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:08.198386 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:08.198643 (+ 257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:08.199190 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:08.199195 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:08.199199 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:08.199611 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:08.199925 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:08.201014 (+ 1089us) server_negotiation.cc:299] Negotiation successful
0811 19:17:08.201199 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 19:17:08.203989 6279 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:17:08.210829 6279 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:17:08.220371 6279 catalog_manager.cc:1347] Generated new cluster ID: c3c21a4c7d0d414d98d4f9344f82b1d1
I20260811 19:17:08.220525 6279 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:17:08.232206 6279 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:17:08.233767 6279 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:17:08.240572 6279 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e29aaf9f688b4c0ebedacdeb0b245709: Generated new TSK 0
I20260811 19:17:08.241815 6279 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:17:08.257478 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 6095
I20260811 19:17:08.258975 6226 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:17:08.358723 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 6095
2026-08-11T19:17:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2487 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786475454804547-3318-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:17:08 dist-test-slave-t7g5 krb5kdc[6294](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:17:08 dist-test-slave-t7g5 krb5kdc[6294](info): set up 2 sockets
Aug 11 19:17:08 dist-test-slave-t7g5 krb5kdc[6294](info): commencing operation
krb5kdc: starting...
W20260811 19:17:09.727377 3318 mini_kdc.cc:121] Time spent starting KDC: real 1.324s user 0.000s sys 0.065s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Aug 11 19:17:09 dist-test-slave-t7g5 krb5kdc[6294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475829, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:17:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:17:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786475454804547-3318-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786475454804547-3318-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 19:17:10.126698 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786475454804547-3318-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786475454804547-3318-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786475454804547-3318-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44645
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786475454804547-3318-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44645
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:17:10.520676 6317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:17:10.521064 6317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:17:10.521210 6317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:17:10.528965 6317 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:17:10.529135 6317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:17:10.529220 6317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:17:10.529310 6317 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:17:10.529376 6317 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:17:10.540132 6317 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786475454804547-3318-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786475454804547-3318-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44645
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786475454804547-3318-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44645
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786475454804547-3318-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:17:10.541810 6317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:17:10.543258 6317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:17:10.556483 6322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:17:10.556469 6324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:17:10.556911 6328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:17:10.557300 6317 server_base.cc:1034] running on GCE node
I20260811 19:17:10.558267 6317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:17:10.559899 6317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:17:10.561158 6317 hybrid_clock.cc:648] HybridClock initialized: now 1786475830561109 us; error 38 us; skew 500 ppm
I20260811 19:17:10.561549 6317 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:17:10 dist-test-slave-t7g5 krb5kdc[6294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475830, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:17:10.565300 6317 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 19:17:10.567041 6317 webserver.cc:460] Webserver started at http://127.0.0.1:33653/ using document root <none> and password file <none>
I20260811 19:17:10.567830 6317 fs_manager.cc:359] Metadata directory not provided
I20260811 19:17:10.567966 6317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:17:10.568287 6317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:17:10.571691 6317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786475454804547-3318-0/minicluster-data/master-0/data/instance:
uuid: "f53184faca5845e9bc7cf721323777c9"
format_stamp: "Formatted at 2026-08-11 19:17:10 on dist-test-slave-t7g5"
server_key: "4dd365ad60c78212ad82972595d7a092"
server_key_iv: "73c692b892d37f86bb89c8817598f1e4"
server_key_version: "encryptionkey@0"
I20260811 19:17:10.572602 6317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786475454804547-3318-0/minicluster-data/master-0/wal/instance:
uuid: "f53184faca5845e9bc7cf721323777c9"
format_stamp: "Formatted at 2026-08-11 19:17:10 on dist-test-slave-t7g5"
server_key: "4dd365ad60c78212ad82972595d7a092"
server_key_iv: "73c692b892d37f86bb89c8817598f1e4"
server_key_version: "encryptionkey@0"
I20260811 19:17:10.579214 6317 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260811 19:17:10.584846 6343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:17:10.586721 6317 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260811 19:17:10.586962 6317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786475454804547-3318-0/minicluster-data/master-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786475454804547-3318-0/minicluster-data/master-0/wal
uuid: "f53184faca5845e9bc7cf721323777c9"
format_stamp: "Formatted at 2026-08-11 19:17:10 on dist-test-slave-t7g5"
server_key: "4dd365ad60c78212ad82972595d7a092"
server_key_iv: "73c692b892d37f86bb89c8817598f1e4"
server_key_version: "encryptionkey@0"
I20260811 19:17:10.587194 6317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786475454804547-3318-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786475454804547-3318-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786475454804547-3318-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:17:10.609768 6317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:17:10.614827 6317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:17:10.615097 6317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:17:10.634933 6317 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44645
I20260811 19:17:10.634979 6440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44645 every 8 connection(s)
I20260811 19:17:10.636390 6317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
I20260811 19:17:10.639941 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 6317
I20260811 19:17:10.640151 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786475454804547-3318-0/minicluster-data/master-0/wal/instance
I20260811 19:17:10.640767 6442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:17:10.640875 3318 external_mini_cluster.cc:1428] Setting key 67f94f874aeda83887a8bd0fbffd8ab8
I20260811 19:17:10.656358 6449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:10.642871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56897 (local address 127.0.0.1:44645)
0811 19:17:10.643802 (+ 931us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:10.643814 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:10.643857 (+ 43us) server_negotiation.cc:407] Connection header received
0811 19:17:10.646860 (+ 3003us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:10.646900 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:10.647672 (+ 772us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:10.648123 (+ 451us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:10.648746 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:10.650310 (+ 1564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:10.651718 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:10.652452 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:10.652622 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:10.653849 (+ 1227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:10.653858 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:10.653876 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:17:10.654484 (+ 608us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:10.654579 (+ 95us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:17:10.654610 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:10.654722 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:10.654972 (+ 250us) server_negotiation.cc:299] Negotiation successful
0811 19:17:10.655262 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":703,"thread_start_us":478,"threads_started":1}
I20260811 19:17:10.657691 6442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f53184faca5845e9bc7cf721323777c9: Bootstrap starting.
I20260811 19:17:10.661799 3318 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:17:10.663373 6442 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f53184faca5845e9bc7cf721323777c9: Neither blocks nor log segments found. Creating new log.
I20260811 19:17:10.665882 6442 log.cc:824] T 00000000000000000000000000000000 P f53184faca5845e9bc7cf721323777c9: Log is configured to *not* fsync() on all Append() calls
I20260811 19:17:10.669378 6442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f53184faca5845e9bc7cf721323777c9: No bootstrap required, opened a new log
I20260811 19:17:10.676482 6449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:10.668796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43589 (local address 127.0.0.1:44645)
0811 19:17:10.668999 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:10.669005 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:10.669370 (+ 365us) server_negotiation.cc:407] Connection header received
0811 19:17:10.669545 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:10.669550 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:10.669641 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:10.669814 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:10.670792 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:10.671955 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:10.673437 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:10.674142 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:10.674288 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:10.675514 (+ 1226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:10.675521 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:10.675527 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:17:10.675553 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:10.675609 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:17:10.675632 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:10.675741 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:10.675975 (+ 234us) server_negotiation.cc:299] Negotiation successful
0811 19:17:10.676090 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 19:17:10.679136 6442 raft_consensus.cc:348] T 00000000000000000000000000000000 P f53184faca5845e9bc7cf721323777c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f53184faca5845e9bc7cf721323777c9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44645 } }
I20260811 19:17:10.679565 6442 raft_consensus.cc:374] T 00000000000000000000000000000000 P f53184faca5845e9bc7cf721323777c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:17:10.679860 6442 raft_consensus.cc:729] T 00000000000000000000000000000000 P f53184faca5845e9bc7cf721323777c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f53184faca5845e9bc7cf721323777c9, State: Initialized, Role: FOLLOWER
I20260811 19:17:10.680866 6442 consensus_queue.cc:260] T 00000000000000000000000000000000 P f53184faca5845e9bc7cf721323777c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f53184faca5845e9bc7cf721323777c9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44645 } }
I20260811 19:17:10.681231 6442 raft_consensus.cc:388] T 00000000000000000000000000000000 P f53184faca5845e9bc7cf721323777c9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:17:10.681389 6442 raft_consensus.cc:482] T 00000000000000000000000000000000 P f53184faca5845e9bc7cf721323777c9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:17:10.681598 6442 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f53184faca5845e9bc7cf721323777c9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:17:10.684810 6442 raft_consensus.cc:504] T 00000000000000000000000000000000 P f53184faca5845e9bc7cf721323777c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f53184faca5845e9bc7cf721323777c9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44645 } }
I20260811 19:17:10.685456 6442 leader_election.cc:302] T 00000000000000000000000000000000 P f53184faca5845e9bc7cf721323777c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f53184faca5845e9bc7cf721323777c9; no voters:
I20260811 19:17:10.686189 6442 leader_election.cc:288] T 00000000000000000000000000000000 P f53184faca5845e9bc7cf721323777c9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:17:10.686405 6457 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f53184faca5845e9bc7cf721323777c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:17:10.687304 6457 raft_consensus.cc:686] T 00000000000000000000000000000000 P f53184faca5845e9bc7cf721323777c9 [term 1 LEADER]: Becoming Leader. State: Replica: f53184faca5845e9bc7cf721323777c9, State: Running, Role: LEADER
I20260811 19:17:10.687959 6457 consensus_queue.cc:237] T 00000000000000000000000000000000 P f53184faca5845e9bc7cf721323777c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f53184faca5845e9bc7cf721323777c9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44645 } }
I20260811 19:17:10.688508 6442 sys_catalog.cc:564] T 00000000000000000000000000000000 P f53184faca5845e9bc7cf721323777c9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:17:10.690948 6458 sys_catalog.cc:455] T 00000000000000000000000000000000 P f53184faca5845e9bc7cf721323777c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f53184faca5845e9bc7cf721323777c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f53184faca5845e9bc7cf721323777c9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44645 } } }
I20260811 19:17:10.691354 6458 sys_catalog.cc:458] T 00000000000000000000000000000000 P f53184faca5845e9bc7cf721323777c9 [sys.catalog]: This master's current role is: LEADER
I20260811 19:17:10.691792 6459 sys_catalog.cc:455] T 00000000000000000000000000000000 P f53184faca5845e9bc7cf721323777c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f53184faca5845e9bc7cf721323777c9. Latest consensus state: current_term: 1 leader_uuid: "f53184faca5845e9bc7cf721323777c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f53184faca5845e9bc7cf721323777c9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44645 } } }
I20260811 19:17:10.692126 6459 sys_catalog.cc:458] T 00000000000000000000000000000000 P f53184faca5845e9bc7cf721323777c9 [sys.catalog]: This master's current role is: LEADER
I20260811 19:17:10.694546 6472 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:17:10.702344 6472 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:17:10.714355 6472 catalog_manager.cc:1347] Generated new cluster ID: 1057e1883e164a1b9acee923bff38dea
I20260811 19:17:10.714493 6472 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:17:10.731148 6472 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:17:10.732730 6472 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:17:10.741120 6472 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f53184faca5845e9bc7cf721323777c9: Generated new TSK 0
I20260811 19:17:10.742424 6472 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:17:10.755728 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 6317
W20260811 19:17:10.757221 6430 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:56897
W20260811 19:17:10.758358 3318 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 6317: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 19:17:10.758623 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 6317
2026-08-11T19:17:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2407 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786475454804547-3318-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:17:10 dist-test-slave-t7g5 krb5kdc[6495](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:17:10 dist-test-slave-t7g5 krb5kdc[6495](info): set up 2 sockets
Aug 11 19:17:10 dist-test-slave-t7g5 krb5kdc[6495](info): commencing operation
krb5kdc: starting...
W20260811 19:17:12.112653 3318 mini_kdc.cc:121] Time spent starting KDC: real 1.300s user 0.000s sys 0.064s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Aug 11 19:17:12 dist-test-slave-t7g5 krb5kdc[6495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475832, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:17:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:17:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786475454804547-3318-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786475454804547-3318-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 19:17:12.540805 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786475454804547-3318-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786475454804547-3318-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786475454804547-3318-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36891
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786475454804547-3318-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36891
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:17:12.947307 6525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:17:12.947942 6525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:17:12.948175 6525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:17:12.956557 6525 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:17:12.956717 6525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:17:12.956804 6525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:17:12.956892 6525 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:17:12.956935 6525 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:17:12.968793 6525 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786475454804547-3318-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786475454804547-3318-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36891
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786475454804547-3318-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36891
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786475454804547-3318-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:17:12.970436 6525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:17:12.971817 6525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:17:12.984553 6533 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:17:12.984658 6538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:17:12.984650 6535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:17:12.986112 6525 server_base.cc:1034] running on GCE node
I20260811 19:17:12.987041 6525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:17:12.988636 6525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:17:12.989869 6525 hybrid_clock.cc:648] HybridClock initialized: now 1786475832989808 us; error 37 us; skew 500 ppm
I20260811 19:17:12.990234 6525 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:17:12 dist-test-slave-t7g5 krb5kdc[6495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475832, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:17:12.993814 6525 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 19:17:12.995450 6525 webserver.cc:460] Webserver started at http://127.0.0.1:33171/ using document root <none> and password file <none>
I20260811 19:17:12.996202 6525 fs_manager.cc:359] Metadata directory not provided
I20260811 19:17:12.996335 6525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:17:12.996731 6525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:17:12.999881 6525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786475454804547-3318-0/minicluster-data/master-0/data/instance:
uuid: "8095c305928c4ab5a88ca5773cb47488"
format_stamp: "Formatted at 2026-08-11 19:17:12 on dist-test-slave-t7g5"
server_key: "045bbe314e6f607840d7f1bd1f86a147"
server_key_iv: "01fcf41fda2174a523d4e6a5297578e8"
server_key_version: "encryptionkey@0"
I20260811 19:17:13.001526 6525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786475454804547-3318-0/minicluster-data/master-0/wal/instance:
uuid: "8095c305928c4ab5a88ca5773cb47488"
format_stamp: "Formatted at 2026-08-11 19:17:12 on dist-test-slave-t7g5"
server_key: "045bbe314e6f607840d7f1bd1f86a147"
server_key_iv: "01fcf41fda2174a523d4e6a5297578e8"
server_key_version: "encryptionkey@0"
I20260811 19:17:13.011149 6525 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.007s sys 0.000s
I20260811 19:17:13.015507 6557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:17:13.017097 6525 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 19:17:13.017318 6525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786475454804547-3318-0/minicluster-data/master-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786475454804547-3318-0/minicluster-data/master-0/wal
uuid: "8095c305928c4ab5a88ca5773cb47488"
format_stamp: "Formatted at 2026-08-11 19:17:12 on dist-test-slave-t7g5"
server_key: "045bbe314e6f607840d7f1bd1f86a147"
server_key_iv: "01fcf41fda2174a523d4e6a5297578e8"
server_key_version: "encryptionkey@0"
I20260811 19:17:13.017539 6525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786475454804547-3318-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786475454804547-3318-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786475454804547-3318-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:17:13.024063 6525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:17:13.028698 6525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:17:13.029012 6525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:17:13.053568 6525 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36891
I20260811 19:17:13.053643 6640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36891 every 8 connection(s)
I20260811 19:17:13.055203 6525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
I20260811 19:17:13.059535 6643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:17:13.064888 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 6525
I20260811 19:17:13.065161 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786475454804547-3318-0/minicluster-data/master-0/wal/instance
I20260811 19:17:13.065744 3318 external_mini_cluster.cc:1428] Setting key 2e71941b64454a526afddb9735ac8b6d
I20260811 19:17:13.072775 6643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8095c305928c4ab5a88ca5773cb47488: Bootstrap starting.
I20260811 19:17:13.075776 6646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:13.068071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59741 (local address 127.0.0.1:36891)
0811 19:17:13.068935 (+ 864us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:13.068948 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:13.068991 (+ 43us) server_negotiation.cc:407] Connection header received
0811 19:17:13.072031 (+ 3040us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:13.072061 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:13.072471 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:13.072828 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:13.074087 (+ 1259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:13.074094 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:13.074109 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:17:13.074134 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:13.074205 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:17:13.074227 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:13.074437 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:13.074547 (+ 110us) server_negotiation.cc:299] Negotiation successful
0811 19:17:13.074755 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":648,"thread_start_us":405,"threads_started":1}
I20260811 19:17:13.076639 6643 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8095c305928c4ab5a88ca5773cb47488: Neither blocks nor log segments found. Creating new log.
I20260811 19:17:13.077780 6643 log.cc:824] T 00000000000000000000000000000000 P 8095c305928c4ab5a88ca5773cb47488: Log is configured to *not* fsync() on all Append() calls
I20260811 19:17:13.080233 3318 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:17:13.081146 6643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8095c305928c4ab5a88ca5773cb47488: No bootstrap required, opened a new log
I20260811 19:17:13.089514 6643 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8095c305928c4ab5a88ca5773cb47488 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8095c305928c4ab5a88ca5773cb47488" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36891 } }
I20260811 19:17:13.090000 6643 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8095c305928c4ab5a88ca5773cb47488 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:17:13.090122 6643 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8095c305928c4ab5a88ca5773cb47488 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8095c305928c4ab5a88ca5773cb47488, State: Initialized, Role: FOLLOWER
I20260811 19:17:13.090462 6646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:13.086635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59173 (local address 127.0.0.1:36891)
0811 19:17:13.086850 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:13.086856 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:13.087362 (+ 506us) server_negotiation.cc:407] Connection header received
0811 19:17:13.087568 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:13.087574 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:13.087647 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:13.087805 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:13.089154 (+ 1349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:13.089161 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:13.089166 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:17:13.089187 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:13.089239 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:17:13.089262 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:13.089408 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:13.090102 (+ 694us) server_negotiation.cc:299] Negotiation successful
0811 19:17:13.090177 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 19:17:13.090942 6643 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8095c305928c4ab5a88ca5773cb47488 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8095c305928c4ab5a88ca5773cb47488" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36891 } }
I20260811 19:17:13.091233 6643 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8095c305928c4ab5a88ca5773cb47488 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:17:13.091394 6643 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8095c305928c4ab5a88ca5773cb47488 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:17:13.091579 6643 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8095c305928c4ab5a88ca5773cb47488 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:17:13.094039 6643 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8095c305928c4ab5a88ca5773cb47488 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8095c305928c4ab5a88ca5773cb47488" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36891 } }
I20260811 19:17:13.094614 6643 leader_election.cc:302] T 00000000000000000000000000000000 P 8095c305928c4ab5a88ca5773cb47488 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8095c305928c4ab5a88ca5773cb47488; no voters:
I20260811 19:17:13.095230 6643 leader_election.cc:288] T 00000000000000000000000000000000 P 8095c305928c4ab5a88ca5773cb47488 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:17:13.095373 6655 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8095c305928c4ab5a88ca5773cb47488 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:17:13.096431 6655 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8095c305928c4ab5a88ca5773cb47488 [term 1 LEADER]: Becoming Leader. State: Replica: 8095c305928c4ab5a88ca5773cb47488, State: Running, Role: LEADER
I20260811 19:17:13.097121 6655 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8095c305928c4ab5a88ca5773cb47488 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8095c305928c4ab5a88ca5773cb47488" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36891 } }
I20260811 19:17:13.097451 6643 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8095c305928c4ab5a88ca5773cb47488 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:17:13.099978 6659 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8095c305928c4ab5a88ca5773cb47488 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8095c305928c4ab5a88ca5773cb47488. Latest consensus state: current_term: 1 leader_uuid: "8095c305928c4ab5a88ca5773cb47488" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8095c305928c4ab5a88ca5773cb47488" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36891 } } }
I20260811 19:17:13.100013 6657 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8095c305928c4ab5a88ca5773cb47488 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8095c305928c4ab5a88ca5773cb47488" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8095c305928c4ab5a88ca5773cb47488" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36891 } } }
I20260811 19:17:13.100456 6659 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8095c305928c4ab5a88ca5773cb47488 [sys.catalog]: This master's current role is: LEADER
I20260811 19:17:13.100453 6657 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8095c305928c4ab5a88ca5773cb47488 [sys.catalog]: This master's current role is: LEADER
I20260811 19:17:13.102329 6672 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:17:13.108525 6672 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:17:13.120450 6672 catalog_manager.cc:1347] Generated new cluster ID: 4e5c48614ca14c5fb06c1c1041c03906
I20260811 19:17:13.120610 6672 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:17:13.149075 6672 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:17:13.149591 6672 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:17:13.157804 6672 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8095c305928c4ab5a88ca5773cb47488: Generated new TSK 0
I20260811 19:17:13.158735 6672 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:17:13.172583 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 6525
W20260811 19:17:13.174192 6634 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:59741
W20260811 19:17:13.175047 3318 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 6525: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 19:17:13.175194 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 6525
2026-08-11T19:17:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2420 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786475454804547-3318-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:17:13 dist-test-slave-t7g5 krb5kdc[6698](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:17:13 dist-test-slave-t7g5 krb5kdc[6698](info): set up 2 sockets
Aug 11 19:17:13 dist-test-slave-t7g5 krb5kdc[6698](info): commencing operation
krb5kdc: starting...
W20260811 19:17:14.535063 3318 mini_kdc.cc:121] Time spent starting KDC: real 1.302s user 0.007s sys 0.063s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Aug 11 19:17:14 dist-test-slave-t7g5 krb5kdc[6698](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475834, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:17:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:17:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786475454804547-3318-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786475454804547-3318-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 19:17:14.921093 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786475454804547-3318-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786475454804547-3318-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786475454804547-3318-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43539
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786475454804547-3318-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43539
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:17:15.299990 6726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:17:15.300449 6726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:17:15.300591 6726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:17:15.309959 6726 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:17:15.310124 6726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:17:15.310179 6726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:17:15.310273 6726 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:17:15.310343 6726 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:17:15.323295 6726 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786475454804547-3318-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786475454804547-3318-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43539
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786475454804547-3318-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43539
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786475454804547-3318-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:17:15.324759 6726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:17:15.326151 6726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:17:15.338352 6740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:17:15.338274 6738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:17:15.338402 6746 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:17:15.339332 6726 server_base.cc:1034] running on GCE node
I20260811 19:17:15.340251 6726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:17:15.341954 6726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:17:15.343181 6726 hybrid_clock.cc:648] HybridClock initialized: now 1786475835343133 us; error 43 us; skew 500 ppm
I20260811 19:17:15.343547 6726 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:17:15 dist-test-slave-t7g5 krb5kdc[6698](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475835, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:17:15.347380 6726 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 19:17:15.349186 6726 webserver.cc:460] Webserver started at http://127.0.0.1:44151/ using document root <none> and password file <none>
I20260811 19:17:15.350039 6726 fs_manager.cc:359] Metadata directory not provided
I20260811 19:17:15.350170 6726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:17:15.350497 6726 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:17:15.354104 6726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786475454804547-3318-0/minicluster-data/master-0/data/instance:
uuid: "ba762b2ebf1e4ec6abaed62b7736e9f3"
format_stamp: "Formatted at 2026-08-11 19:17:15 on dist-test-slave-t7g5"
server_key: "36d97378bec25638992e56bb4f48f2d0"
server_key_iv: "38fe97a535b5a39b8a604e6830246326"
server_key_version: "encryptionkey@0"
I20260811 19:17:15.354955 6726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786475454804547-3318-0/minicluster-data/master-0/wal/instance:
uuid: "ba762b2ebf1e4ec6abaed62b7736e9f3"
format_stamp: "Formatted at 2026-08-11 19:17:15 on dist-test-slave-t7g5"
server_key: "36d97378bec25638992e56bb4f48f2d0"
server_key_iv: "38fe97a535b5a39b8a604e6830246326"
server_key_version: "encryptionkey@0"
I20260811 19:17:15.360704 6726 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 19:17:15.365516 6755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:17:15.367169 6726 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 19:17:15.367381 6726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786475454804547-3318-0/minicluster-data/master-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786475454804547-3318-0/minicluster-data/master-0/wal
uuid: "ba762b2ebf1e4ec6abaed62b7736e9f3"
format_stamp: "Formatted at 2026-08-11 19:17:15 on dist-test-slave-t7g5"
server_key: "36d97378bec25638992e56bb4f48f2d0"
server_key_iv: "38fe97a535b5a39b8a604e6830246326"
server_key_version: "encryptionkey@0"
I20260811 19:17:15.367601 6726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786475454804547-3318-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786475454804547-3318-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786475454804547-3318-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:17:15.390210 6726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:17:15.395288 6726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:17:15.395725 6726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:17:15.416164 6726 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43539
I20260811 19:17:15.416224 6846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43539 every 8 connection(s)
I20260811 19:17:15.417603 6726 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
I20260811 19:17:15.421509 6847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:17:15.421773 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 6726
I20260811 19:17:15.421957 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786475454804547-3318-0/minicluster-data/master-0/wal/instance
I20260811 19:17:15.422453 3318 external_mini_cluster.cc:1428] Setting key 1cf3595294e87c12b3047c916562d8fa
I20260811 19:17:15.438746 6847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba762b2ebf1e4ec6abaed62b7736e9f3: Bootstrap starting.
Aug 11 19:17:15 dist-test-slave-t7g5 krb5kdc[6698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475834, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 19:17:15.442471 6847 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ba762b2ebf1e4ec6abaed62b7736e9f3: Neither blocks nor log segments found. Creating new log.
I20260811 19:17:15.444054 6847 log.cc:824] T 00000000000000000000000000000000 P ba762b2ebf1e4ec6abaed62b7736e9f3: Log is configured to *not* fsync() on all Append() calls
I20260811 19:17:15.448081 6847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba762b2ebf1e4ec6abaed62b7736e9f3: No bootstrap required, opened a new log
I20260811 19:17:15.452593 6857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:15.424491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.70:47675 (local address 127.0.0.1:43539)
0811 19:17:15.425332 (+ 841us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:15.425349 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:15.425403 (+ 54us) server_negotiation.cc:407] Connection header received
0811 19:17:15.429497 (+ 4094us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:15.429538 (+ 41us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:15.430265 (+ 727us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:15.430837 (+ 572us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:15.432566 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:15.434712 (+ 2146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:15.436220 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:15.437425 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:15.437638 (+ 213us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:15.442269 (+ 4631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:15.442283 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:15.442305 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:17:15.442374 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:15.447162 (+ 4788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:15.448150 (+ 988us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:15.448159 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:15.448168 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:15.448393 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:15.448921 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:15.448927 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:15.448930 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:15.449681 (+ 751us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:15.450047 (+ 366us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:15.450755 (+ 708us) server_negotiation.cc:299] Negotiation successful
0811 19:17:15.451152 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":335,"threads_started":1}
I20260811 19:17:15.457031 3318 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:17:15.457939 6847 raft_consensus.cc:348] T 00000000000000000000000000000000 P ba762b2ebf1e4ec6abaed62b7736e9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba762b2ebf1e4ec6abaed62b7736e9f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43539 } }
I20260811 19:17:15.458313 6847 raft_consensus.cc:374] T 00000000000000000000000000000000 P ba762b2ebf1e4ec6abaed62b7736e9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:17:15.458418 6847 raft_consensus.cc:729] T 00000000000000000000000000000000 P ba762b2ebf1e4ec6abaed62b7736e9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba762b2ebf1e4ec6abaed62b7736e9f3, State: Initialized, Role: FOLLOWER
I20260811 19:17:15.459002 6847 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba762b2ebf1e4ec6abaed62b7736e9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba762b2ebf1e4ec6abaed62b7736e9f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43539 } }
I20260811 19:17:15.459230 6847 raft_consensus.cc:388] T 00000000000000000000000000000000 P ba762b2ebf1e4ec6abaed62b7736e9f3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:17:15.459372 6847 raft_consensus.cc:482] T 00000000000000000000000000000000 P ba762b2ebf1e4ec6abaed62b7736e9f3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:17:15.459523 6847 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ba762b2ebf1e4ec6abaed62b7736e9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:17:15.462298 6847 raft_consensus.cc:504] T 00000000000000000000000000000000 P ba762b2ebf1e4ec6abaed62b7736e9f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba762b2ebf1e4ec6abaed62b7736e9f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43539 } }
I20260811 19:17:15.462997 6847 leader_election.cc:302] T 00000000000000000000000000000000 P ba762b2ebf1e4ec6abaed62b7736e9f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ba762b2ebf1e4ec6abaed62b7736e9f3; no voters:
I20260811 19:17:15.464211 6847 leader_election.cc:288] T 00000000000000000000000000000000 P ba762b2ebf1e4ec6abaed62b7736e9f3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:17:15.464327 6871 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ba762b2ebf1e4ec6abaed62b7736e9f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:17:15.467170 6847 sys_catalog.cc:564] T 00000000000000000000000000000000 P ba762b2ebf1e4ec6abaed62b7736e9f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:17:15.471443 6871 raft_consensus.cc:686] T 00000000000000000000000000000000 P ba762b2ebf1e4ec6abaed62b7736e9f3 [term 1 LEADER]: Becoming Leader. State: Replica: ba762b2ebf1e4ec6abaed62b7736e9f3, State: Running, Role: LEADER
I20260811 19:17:15.472244 6871 consensus_queue.cc:237] T 00000000000000000000000000000000 P ba762b2ebf1e4ec6abaed62b7736e9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba762b2ebf1e4ec6abaed62b7736e9f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43539 } }
I20260811 19:17:15.476557 6876 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba762b2ebf1e4ec6abaed62b7736e9f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ba762b2ebf1e4ec6abaed62b7736e9f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba762b2ebf1e4ec6abaed62b7736e9f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43539 } } }
I20260811 19:17:15.477268 6876 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba762b2ebf1e4ec6abaed62b7736e9f3 [sys.catalog]: This master's current role is: LEADER
I20260811 19:17:15.478232 6877 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba762b2ebf1e4ec6abaed62b7736e9f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba762b2ebf1e4ec6abaed62b7736e9f3. Latest consensus state: current_term: 1 leader_uuid: "ba762b2ebf1e4ec6abaed62b7736e9f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba762b2ebf1e4ec6abaed62b7736e9f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43539 } } }
I20260811 19:17:15.478608 6877 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba762b2ebf1e4ec6abaed62b7736e9f3 [sys.catalog]: This master's current role is: LEADER
I20260811 19:17:15.484681 6857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:15.462546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.70:46621 (local address 127.0.0.1:43539)
0811 19:17:15.462699 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:15.462703 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:15.463249 (+ 546us) server_negotiation.cc:407] Connection header received
0811 19:17:15.463572 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:15.463580 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:15.463705 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:15.463848 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:15.466045 (+ 2197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:15.467304 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:15.470004 (+ 2700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:15.470849 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:15.470973 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:15.474939 (+ 3966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:15.474946 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:15.474952 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:17:15.475005 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:15.479727 (+ 4722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:15.480908 (+ 1181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:15.480915 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:15.480919 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:15.481098 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:15.481819 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:15.481825 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:15.481829 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:15.482256 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:15.483846 (+ 1590us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:15.484043 (+ 197us) server_negotiation.cc:299] Negotiation successful
0811 19:17:15.484262 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 19:17:15.486817 6896 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:17:15.492728 6896 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:17:15.504402 6896 catalog_manager.cc:1347] Generated new cluster ID: e25d18c8fcf042c7b3600799699a2fef
I20260811 19:17:15.504549 6896 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:17:15.542435 6896 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:17:15.544389 6896 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:17:15.553628 6896 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ba762b2ebf1e4ec6abaed62b7736e9f3: Generated new TSK 0
I20260811 19:17:15.554785 6896 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:17:15.574839 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 6726
I20260811 19:17:15.577105 6837 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:17:15.681903 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 6726
W20260811 19:17:15.712003 6692 connection.cc:535] client connection to 127.0.0.1:43539 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43539): Connection reset by peer (error 104)
2026-08-11T19:17:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2500 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786475454804547-3318-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:17:15 dist-test-slave-t7g5 krb5kdc[6916](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:17:15 dist-test-slave-t7g5 krb5kdc[6916](info): set up 2 sockets
Aug 11 19:17:15 dist-test-slave-t7g5 krb5kdc[6916](info): commencing operation
krb5kdc: starting...
W20260811 19:17:17.018682 3318 mini_kdc.cc:121] Time spent starting KDC: real 1.287s user 0.003s sys 0.066s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Aug 11 19:17:17 dist-test-slave-t7g5 krb5kdc[6916](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475837, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:17:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:17:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786475454804547-3318-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786475454804547-3318-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 19:17:17.413200 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786475454804547-3318-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786475454804547-3318-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786475454804547-3318-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41961
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786475454804547-3318-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41961
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:17:17.810735 6948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:17:17.811080 6948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:17:17.811223 6948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:17:17.818836 6948 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:17:17.819003 6948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:17:17.819082 6948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:17:17.819166 6948 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:17:17.819228 6948 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:17:17.831344 6948 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786475454804547-3318-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786475454804547-3318-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41961
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786475454804547-3318-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786475454804547-3318-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:17:17.833093 6948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:17:17.834518 6948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:17:17.846975 6956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:17:17.847043 6963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:17:17.847070 6960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:17:17.848147 6948 server_base.cc:1034] running on GCE node
I20260811 19:17:17.849455 6948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:17:17.851305 6948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:17:17.852582 6948 hybrid_clock.cc:648] HybridClock initialized: now 1786475837852516 us; error 57 us; skew 500 ppm
I20260811 19:17:17.852979 6948 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:17:17 dist-test-slave-t7g5 krb5kdc[6916](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475837, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:17:17.856563 6948 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 19:17:17.858255 6948 webserver.cc:460] Webserver started at http://127.0.0.1:41347/ using document root <none> and password file <none>
I20260811 19:17:17.858999 6948 fs_manager.cc:359] Metadata directory not provided
I20260811 19:17:17.859110 6948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:17:17.859406 6948 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:17:17.863749 6948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786475454804547-3318-0/minicluster-data/master-0/data/instance:
uuid: "c24d27edb8eb4cf1818c20137be66a70"
format_stamp: "Formatted at 2026-08-11 19:17:17 on dist-test-slave-t7g5"
server_key: "32e2a57b10e8fb7ac97e682530b3eed7"
server_key_iv: "b7661a6aab3520e773db88cc1858b949"
server_key_version: "encryptionkey@0"
I20260811 19:17:17.864802 6948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786475454804547-3318-0/minicluster-data/master-0/wal/instance:
uuid: "c24d27edb8eb4cf1818c20137be66a70"
format_stamp: "Formatted at 2026-08-11 19:17:17 on dist-test-slave-t7g5"
server_key: "32e2a57b10e8fb7ac97e682530b3eed7"
server_key_iv: "b7661a6aab3520e773db88cc1858b949"
server_key_version: "encryptionkey@0"
I20260811 19:17:17.871425 6948 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.008s
I20260811 19:17:17.875581 6975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:17:17.877163 6948 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 19:17:17.877372 6948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786475454804547-3318-0/minicluster-data/master-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786475454804547-3318-0/minicluster-data/master-0/wal
uuid: "c24d27edb8eb4cf1818c20137be66a70"
format_stamp: "Formatted at 2026-08-11 19:17:17 on dist-test-slave-t7g5"
server_key: "32e2a57b10e8fb7ac97e682530b3eed7"
server_key_iv: "b7661a6aab3520e773db88cc1858b949"
server_key_version: "encryptionkey@0"
I20260811 19:17:17.877596 6948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786475454804547-3318-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786475454804547-3318-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786475454804547-3318-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:17:17.906011 6948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:17:17.912102 6948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:17:17.912431 6948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:17:17.934192 6948 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41961
I20260811 19:17:17.934242 7078 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41961 every 8 connection(s)
I20260811 19:17:17.935685 6948 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
I20260811 19:17:17.939522 7079 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:17:17.945031 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 6948
I20260811 19:17:17.945235 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786475454804547-3318-0/minicluster-data/master-0/wal/instance
I20260811 19:17:17.945713 3318 external_mini_cluster.cc:1428] Setting key 18c88f513ac2d150e354420f1a99c4fd
I20260811 19:17:17.953532 7079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c24d27edb8eb4cf1818c20137be66a70: Bootstrap starting.
I20260811 19:17:17.957331 7079 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c24d27edb8eb4cf1818c20137be66a70: Neither blocks nor log segments found. Creating new log.
I20260811 19:17:17.958724 7079 log.cc:824] T 00000000000000000000000000000000 P c24d27edb8eb4cf1818c20137be66a70: Log is configured to *not* fsync() on all Append() calls
I20260811 19:17:17.963027 7083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:17.948374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.70:42319 (local address 127.0.0.1:41961)
0811 19:17:17.949482 (+ 1108us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:17.949494 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:17.949537 (+ 43us) server_negotiation.cc:407] Connection header received
0811 19:17:17.951906 (+ 2369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:17.951955 (+ 49us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:17.952433 (+ 478us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:17.952779 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:17.953507 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:17.954939 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:17.956882 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:17.957712 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:17.957880 (+ 168us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:17.959200 (+ 1320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:17.959209 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:17.959231 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:17:17.960118 (+ 887us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:17.960233 (+ 115us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:17:17.960270 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:17.960460 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:17.961304 (+ 844us) server_negotiation.cc:299] Negotiation successful
0811 19:17:17.961538 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":850,"thread_start_us":373,"threads_started":1}
I20260811 19:17:17.963294 7079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c24d27edb8eb4cf1818c20137be66a70: No bootstrap required, opened a new log
I20260811 19:17:17.968230 3318 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:17:17.975329 7079 raft_consensus.cc:348] T 00000000000000000000000000000000 P c24d27edb8eb4cf1818c20137be66a70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c24d27edb8eb4cf1818c20137be66a70" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41961 } }
I20260811 19:17:17.975906 7079 raft_consensus.cc:374] T 00000000000000000000000000000000 P c24d27edb8eb4cf1818c20137be66a70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:17:17.976050 7079 raft_consensus.cc:729] T 00000000000000000000000000000000 P c24d27edb8eb4cf1818c20137be66a70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c24d27edb8eb4cf1818c20137be66a70, State: Initialized, Role: FOLLOWER
I20260811 19:17:17.976759 7079 consensus_queue.cc:260] T 00000000000000000000000000000000 P c24d27edb8eb4cf1818c20137be66a70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c24d27edb8eb4cf1818c20137be66a70" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41961 } }
I20260811 19:17:17.977075 7079 raft_consensus.cc:388] T 00000000000000000000000000000000 P c24d27edb8eb4cf1818c20137be66a70 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:17:17.977231 7079 raft_consensus.cc:482] T 00000000000000000000000000000000 P c24d27edb8eb4cf1818c20137be66a70 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:17:17.977429 7079 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c24d27edb8eb4cf1818c20137be66a70 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:17:17.980216 7079 raft_consensus.cc:504] T 00000000000000000000000000000000 P c24d27edb8eb4cf1818c20137be66a70 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c24d27edb8eb4cf1818c20137be66a70" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41961 } }
I20260811 19:17:17.980777 7079 leader_election.cc:302] T 00000000000000000000000000000000 P c24d27edb8eb4cf1818c20137be66a70 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c24d27edb8eb4cf1818c20137be66a70; no voters:
I20260811 19:17:17.981464 7079 leader_election.cc:288] T 00000000000000000000000000000000 P c24d27edb8eb4cf1818c20137be66a70 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:17:17.981592 7098 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c24d27edb8eb4cf1818c20137be66a70 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:17:17.982260 7083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:17.974990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.70:33447 (local address 127.0.0.1:41961)
0811 19:17:17.975208 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:17.975214 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:17.975555 (+ 341us) server_negotiation.cc:407] Connection header received
0811 19:17:17.975711 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:17.975716 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:17.975805 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:17.975937 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:17.976571 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:17.977708 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:17.979184 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:17.979890 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:17.980017 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:17.981196 (+ 1179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:17.981201 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:17.981205 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:17:17.981223 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:17.981288 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:17:17.981306 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:17.981417 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:17.981757 (+ 340us) server_negotiation.cc:299] Negotiation successful
0811 19:17:17.981815 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 19:17:17.983281 7098 raft_consensus.cc:686] T 00000000000000000000000000000000 P c24d27edb8eb4cf1818c20137be66a70 [term 1 LEADER]: Becoming Leader. State: Replica: c24d27edb8eb4cf1818c20137be66a70, State: Running, Role: LEADER
I20260811 19:17:17.983966 7098 consensus_queue.cc:237] T 00000000000000000000000000000000 P c24d27edb8eb4cf1818c20137be66a70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c24d27edb8eb4cf1818c20137be66a70" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41961 } }
I20260811 19:17:17.984158 7079 sys_catalog.cc:564] T 00000000000000000000000000000000 P c24d27edb8eb4cf1818c20137be66a70 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:17:17.986999 7103 sys_catalog.cc:455] T 00000000000000000000000000000000 P c24d27edb8eb4cf1818c20137be66a70 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c24d27edb8eb4cf1818c20137be66a70. Latest consensus state: current_term: 1 leader_uuid: "c24d27edb8eb4cf1818c20137be66a70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c24d27edb8eb4cf1818c20137be66a70" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41961 } } }
I20260811 19:17:17.987408 7103 sys_catalog.cc:458] T 00000000000000000000000000000000 P c24d27edb8eb4cf1818c20137be66a70 [sys.catalog]: This master's current role is: LEADER
I20260811 19:17:17.987996 7102 sys_catalog.cc:455] T 00000000000000000000000000000000 P c24d27edb8eb4cf1818c20137be66a70 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c24d27edb8eb4cf1818c20137be66a70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c24d27edb8eb4cf1818c20137be66a70" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41961 } } }
I20260811 19:17:17.988334 7102 sys_catalog.cc:458] T 00000000000000000000000000000000 P c24d27edb8eb4cf1818c20137be66a70 [sys.catalog]: This master's current role is: LEADER
I20260811 19:17:17.994482 7112 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:17:18.000224 7112 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:17:18.013412 7112 catalog_manager.cc:1347] Generated new cluster ID: a4473a03e5694087ad9f7150cd24d723
I20260811 19:17:18.013561 7112 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:17:18.056475 7112 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:17:18.058718 7112 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:17:18.075405 7112 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c24d27edb8eb4cf1818c20137be66a70: Generated new TSK 0
I20260811 19:17:18.076323 7112 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:17:18.085821 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 6948
W20260811 19:17:18.087440 7070 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.70:42319
W20260811 19:17:18.088186 3318 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 6948: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 19:17:18.088326 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 6948
W20260811 19:17:18.115841 6912 connection.cc:535] client connection to 127.0.0.1:41961 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41961): Connection reset by peer (error 104)
2026-08-11T19:17:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2403 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786475454804547-3318-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:17:18 dist-test-slave-t7g5 krb5kdc[7138](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:17:18 dist-test-slave-t7g5 krb5kdc[7138](info): set up 2 sockets
Aug 11 19:17:18 dist-test-slave-t7g5 krb5kdc[7138](info): commencing operation
krb5kdc: starting...
W20260811 19:17:19.439258 3318 mini_kdc.cc:121] Time spent starting KDC: real 1.303s user 0.000s sys 0.066s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Aug 11 19:17:19 dist-test-slave-t7g5 krb5kdc[7138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475839, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:17:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:17:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786475454804547-3318-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786475454804547-3318-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 19:17:19.833745 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786475454804547-3318-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786475454804547-3318-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786475454804547-3318-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44783
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786475454804547-3318-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44783
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:17:20.234074 7165 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:17:20.234422 7165 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:17:20.234526 7165 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:17:20.242913 7165 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:17:20.243125 7165 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:17:20.243216 7165 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:17:20.243320 7165 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:17:20.243377 7165 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:17:20.256796 7165 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786475454804547-3318-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786475454804547-3318-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44783
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786475454804547-3318-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44783
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786475454804547-3318-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:17:20.258322 7165 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:17:20.260205 7165 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:17:20.275173 7173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:17:20.275177 7176 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:17:20.275418 7174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:17:20.275995 7165 server_base.cc:1034] running on GCE node
I20260811 19:17:20.276891 7165 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:17:20.278690 7165 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:17:20.279946 7165 hybrid_clock.cc:648] HybridClock initialized: now 1786475840279884 us; error 50 us; skew 500 ppm
I20260811 19:17:20.280350 7165 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:17:20 dist-test-slave-t7g5 krb5kdc[7138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475840, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:17:20.284198 7165 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 19:17:20.285923 7165 webserver.cc:460] Webserver started at http://127.0.0.1:45061/ using document root <none> and password file <none>
I20260811 19:17:20.286711 7165 fs_manager.cc:359] Metadata directory not provided
I20260811 19:17:20.286871 7165 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:17:20.287230 7165 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:17:20.290678 7165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786475454804547-3318-0/minicluster-data/master-0/data/instance:
uuid: "65cd40eb2d7f460bb48054eb46281848"
format_stamp: "Formatted at 2026-08-11 19:17:20 on dist-test-slave-t7g5"
server_key: "e54bc0f803107bb1bb4543514b0a55d7"
server_key_iv: "433943935e14b8e8e1b298c6064133f6"
server_key_version: "encryptionkey@0"
I20260811 19:17:20.291502 7165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786475454804547-3318-0/minicluster-data/master-0/wal/instance:
uuid: "65cd40eb2d7f460bb48054eb46281848"
format_stamp: "Formatted at 2026-08-11 19:17:20 on dist-test-slave-t7g5"
server_key: "e54bc0f803107bb1bb4543514b0a55d7"
server_key_iv: "433943935e14b8e8e1b298c6064133f6"
server_key_version: "encryptionkey@0"
I20260811 19:17:20.298087 7165 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260811 19:17:20.303068 7187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:17:20.304795 7165 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.001s
I20260811 19:17:20.305033 7165 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786475454804547-3318-0/minicluster-data/master-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786475454804547-3318-0/minicluster-data/master-0/wal
uuid: "65cd40eb2d7f460bb48054eb46281848"
format_stamp: "Formatted at 2026-08-11 19:17:20 on dist-test-slave-t7g5"
server_key: "e54bc0f803107bb1bb4543514b0a55d7"
server_key_iv: "433943935e14b8e8e1b298c6064133f6"
server_key_version: "encryptionkey@0"
I20260811 19:17:20.305256 7165 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786475454804547-3318-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786475454804547-3318-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786475454804547-3318-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:17:20.311757 7165 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:17:20.316421 7165 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:17:20.316740 7165 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:17:20.337970 7165 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44783
I20260811 19:17:20.338011 7289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44783 every 8 connection(s)
I20260811 19:17:20.339442 7165 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
I20260811 19:17:20.343456 7290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:17:20.344825 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 7165
I20260811 19:17:20.344995 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786475454804547-3318-0/minicluster-data/master-0/wal/instance
I20260811 19:17:20.345439 3318 external_mini_cluster.cc:1428] Setting key cf61ead2293a519b916f697b61207ffd
I20260811 19:17:20.353843 7294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:20.347526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.70:46207 (local address 127.0.0.1:44783)
0811 19:17:20.348390 (+ 864us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:20.348403 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:20.348449 (+ 46us) server_negotiation.cc:407] Connection header received
0811 19:17:20.350151 (+ 1702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:20.350180 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:20.350579 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:20.350946 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:20.352126 (+ 1180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:20.352134 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:20.352148 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:17:20.352166 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:20.352241 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:17:20.352272 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:20.352466 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:20.352578 (+ 112us) server_negotiation.cc:299] Negotiation successful
0811 19:17:20.352783 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":618,"thread_start_us":386,"threads_started":1}
I20260811 19:17:20.357960 7290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65cd40eb2d7f460bb48054eb46281848: Bootstrap starting.
I20260811 19:17:20.358548 3318 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:17:20.362877 7290 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 65cd40eb2d7f460bb48054eb46281848: Neither blocks nor log segments found. Creating new log.
I20260811 19:17:20.365096 7290 log.cc:824] T 00000000000000000000000000000000 P 65cd40eb2d7f460bb48054eb46281848: Log is configured to *not* fsync() on all Append() calls
I20260811 19:17:20.367269 7294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:20.363375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.70:49035 (local address 127.0.0.1:44783)
0811 19:17:20.363596 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:20.363604 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:20.364414 (+ 810us) server_negotiation.cc:407] Connection header received
0811 19:17:20.364628 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:20.364633 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:20.364692 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:20.364784 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:20.366231 (+ 1447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:20.366237 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:20.366247 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:17:20.366274 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:20.366345 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:17:20.366370 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:20.366619 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:20.366893 (+ 274us) server_negotiation.cc:299] Negotiation successful
0811 19:17:20.366970 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 19:17:20.369143 7290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65cd40eb2d7f460bb48054eb46281848: No bootstrap required, opened a new log
I20260811 19:17:20.377501 7290 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65cd40eb2d7f460bb48054eb46281848 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65cd40eb2d7f460bb48054eb46281848" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44783 } }
I20260811 19:17:20.378029 7290 raft_consensus.cc:374] T 00000000000000000000000000000000 P 65cd40eb2d7f460bb48054eb46281848 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:17:20.378135 7290 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65cd40eb2d7f460bb48054eb46281848 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65cd40eb2d7f460bb48054eb46281848, State: Initialized, Role: FOLLOWER
I20260811 19:17:20.378801 7290 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65cd40eb2d7f460bb48054eb46281848 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65cd40eb2d7f460bb48054eb46281848" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44783 } }
I20260811 19:17:20.379040 7290 raft_consensus.cc:388] T 00000000000000000000000000000000 P 65cd40eb2d7f460bb48054eb46281848 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:17:20.379176 7290 raft_consensus.cc:482] T 00000000000000000000000000000000 P 65cd40eb2d7f460bb48054eb46281848 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:17:20.379356 7290 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65cd40eb2d7f460bb48054eb46281848 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:17:20.381700 7290 raft_consensus.cc:504] T 00000000000000000000000000000000 P 65cd40eb2d7f460bb48054eb46281848 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65cd40eb2d7f460bb48054eb46281848" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44783 } }
I20260811 19:17:20.382290 7290 leader_election.cc:302] T 00000000000000000000000000000000 P 65cd40eb2d7f460bb48054eb46281848 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 65cd40eb2d7f460bb48054eb46281848; no voters:
I20260811 19:17:20.383108 7290 leader_election.cc:288] T 00000000000000000000000000000000 P 65cd40eb2d7f460bb48054eb46281848 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:17:20.383491 7307 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 65cd40eb2d7f460bb48054eb46281848 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:17:20.384464 7307 raft_consensus.cc:686] T 00000000000000000000000000000000 P 65cd40eb2d7f460bb48054eb46281848 [term 1 LEADER]: Becoming Leader. State: Replica: 65cd40eb2d7f460bb48054eb46281848, State: Running, Role: LEADER
I20260811 19:17:20.385059 7307 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65cd40eb2d7f460bb48054eb46281848 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65cd40eb2d7f460bb48054eb46281848" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44783 } }
I20260811 19:17:20.385565 7290 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65cd40eb2d7f460bb48054eb46281848 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:17:20.388597 7314 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65cd40eb2d7f460bb48054eb46281848 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65cd40eb2d7f460bb48054eb46281848. Latest consensus state: current_term: 1 leader_uuid: "65cd40eb2d7f460bb48054eb46281848" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65cd40eb2d7f460bb48054eb46281848" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44783 } } }
I20260811 19:17:20.388993 7314 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65cd40eb2d7f460bb48054eb46281848 [sys.catalog]: This master's current role is: LEADER
I20260811 19:17:20.389462 7309 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65cd40eb2d7f460bb48054eb46281848 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "65cd40eb2d7f460bb48054eb46281848" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65cd40eb2d7f460bb48054eb46281848" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44783 } } }
I20260811 19:17:20.389808 7309 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65cd40eb2d7f460bb48054eb46281848 [sys.catalog]: This master's current role is: LEADER
I20260811 19:17:20.390128 7330 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:17:20.397271 7330 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:17:20.409183 7330 catalog_manager.cc:1347] Generated new cluster ID: d27efa47e46b47f9bc08cc12ce9ab907
I20260811 19:17:20.409308 7330 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:17:20.438742 7330 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:17:20.439245 7330 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:17:20.446099 7330 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 65cd40eb2d7f460bb48054eb46281848: Generated new TSK 0
I20260811 19:17:20.446978 7330 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:17:20.462011 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 7165
W20260811 19:17:20.463189 7280 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.70:46207
W20260811 19:17:20.463994 3318 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 7165: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 19:17:20.464178 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 7165
2026-08-11T19:17:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2375 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786475454804547-3318-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:17:20 dist-test-slave-t7g5 krb5kdc[7361](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:17:20 dist-test-slave-t7g5 krb5kdc[7361](info): set up 2 sockets
Aug 11 19:17:20 dist-test-slave-t7g5 krb5kdc[7361](info): commencing operation
krb5kdc: starting...
W20260811 19:17:21.797935 3318 mini_kdc.cc:121] Time spent starting KDC: real 1.287s user 0.005s sys 0.055s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Aug 11 19:17:22 dist-test-slave-t7g5 krb5kdc[7361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475842, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:17:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:17:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786475454804547-3318-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786475454804547-3318-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 19:17:22.210378 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786475454804547-3318-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786475454804547-3318-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786475454804547-3318-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44549
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786475454804547-3318-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44549
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:17:22.615407 7393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:17:22.615885 7393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:17:22.616058 7393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:17:22.624801 7393 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:17:22.624984 7393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:17:22.625074 7393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:17:22.625169 7393 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:17:22.625239 7393 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:17:22.637279 7393 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786475454804547-3318-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786475454804547-3318-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44549
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786475454804547-3318-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44549
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786475454804547-3318-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:17:22.638819 7393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:17:22.640214 7393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:17:22.652951 7402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:17:22.653164 7406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:17:22.653144 7403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:17:22.654150 7393 server_base.cc:1034] running on GCE node
I20260811 19:17:22.655067 7393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:17:22.656808 7393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:17:22.660207 7393 hybrid_clock.cc:648] HybridClock initialized: now 1786475842660118 us; error 164 us; skew 500 ppm
I20260811 19:17:22.660714 7393 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:17:22 dist-test-slave-t7g5 krb5kdc[7361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475842, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:17:22.665226 7393 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 19:17:22.667709 7393 webserver.cc:460] Webserver started at http://127.0.0.1:36949/ using document root <none> and password file <none>
I20260811 19:17:22.668785 7393 fs_manager.cc:359] Metadata directory not provided
I20260811 19:17:22.668931 7393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:17:22.669330 7393 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:17:22.673601 7393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786475454804547-3318-0/minicluster-data/master-0/data/instance:
uuid: "721dc63a2c7a49f0886a7016f935c642"
format_stamp: "Formatted at 2026-08-11 19:17:22 on dist-test-slave-t7g5"
server_key: "61086d1a3a193ddbf7c7ffe7dd9f05d7"
server_key_iv: "c7549528a845455d6c4029a246bde167"
server_key_version: "encryptionkey@0"
I20260811 19:17:22.674732 7393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786475454804547-3318-0/minicluster-data/master-0/wal/instance:
uuid: "721dc63a2c7a49f0886a7016f935c642"
format_stamp: "Formatted at 2026-08-11 19:17:22 on dist-test-slave-t7g5"
server_key: "61086d1a3a193ddbf7c7ffe7dd9f05d7"
server_key_iv: "c7549528a845455d6c4029a246bde167"
server_key_version: "encryptionkey@0"
I20260811 19:17:22.682317 7393 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260811 19:17:22.688090 7417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:17:22.690542 7393 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260811 19:17:22.690825 7393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786475454804547-3318-0/minicluster-data/master-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786475454804547-3318-0/minicluster-data/master-0/wal
uuid: "721dc63a2c7a49f0886a7016f935c642"
format_stamp: "Formatted at 2026-08-11 19:17:22 on dist-test-slave-t7g5"
server_key: "61086d1a3a193ddbf7c7ffe7dd9f05d7"
server_key_iv: "c7549528a845455d6c4029a246bde167"
server_key_version: "encryptionkey@0"
I20260811 19:17:22.691092 7393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786475454804547-3318-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786475454804547-3318-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786475454804547-3318-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:17:22.724841 7393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:17:22.730686 7393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:17:22.731070 7393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:17:22.754851 7393 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44549
I20260811 19:17:22.754890 7518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44549 every 8 connection(s)
I20260811 19:17:22.756697 7393 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
I20260811 19:17:22.762285 7519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:17:22.767103 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 7393
I20260811 19:17:22.767313 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786475454804547-3318-0/minicluster-data/master-0/wal/instance
I20260811 19:17:22.767771 3318 external_mini_cluster.cc:1428] Setting key 4b224730103317f1ddedd5cdf7b52ffd
I20260811 19:17:22.774996 7519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 721dc63a2c7a49f0886a7016f935c642: Bootstrap starting.
I20260811 19:17:22.778549 7519 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 721dc63a2c7a49f0886a7016f935c642: Neither blocks nor log segments found. Creating new log.
I20260811 19:17:22.779654 7519 log.cc:824] T 00000000000000000000000000000000 P 721dc63a2c7a49f0886a7016f935c642: Log is configured to *not* fsync() on all Append() calls
I20260811 19:17:22.783552 7519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 721dc63a2c7a49f0886a7016f935c642: No bootstrap required, opened a new log
Aug 11 19:17:22 dist-test-slave-t7g5 krb5kdc[7361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475842, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 19:17:22.794157 7523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:22.770094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.70:46959 (local address 127.0.0.1:44549)
0811 19:17:22.771008 (+ 914us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:22.771021 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:22.771073 (+ 52us) server_negotiation.cc:407] Connection header received
0811 19:17:22.773948 (+ 2875us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:22.773981 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:22.774571 (+ 590us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:22.775023 (+ 452us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:22.776750 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:22.778195 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:22.779579 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:22.780479 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:22.780698 (+ 219us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:22.785744 (+ 5046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:22.785757 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:22.785783 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:17:22.785828 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:22.789184 (+ 3356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:22.790194 (+ 1010us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:22.790201 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:22.790208 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:22.790419 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:22.790901 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:22.790905 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:22.790907 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:22.791559 (+ 652us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:22.791919 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:22.792469 (+ 550us) server_negotiation.cc:299] Negotiation successful
0811 19:17:22.792874 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":652,"thread_start_us":386,"threads_started":1}
I20260811 19:17:22.794713 7519 raft_consensus.cc:348] T 00000000000000000000000000000000 P 721dc63a2c7a49f0886a7016f935c642 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "721dc63a2c7a49f0886a7016f935c642" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44549 } }
I20260811 19:17:22.795526 7519 raft_consensus.cc:374] T 00000000000000000000000000000000 P 721dc63a2c7a49f0886a7016f935c642 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:17:22.795702 7519 raft_consensus.cc:729] T 00000000000000000000000000000000 P 721dc63a2c7a49f0886a7016f935c642 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 721dc63a2c7a49f0886a7016f935c642, State: Initialized, Role: FOLLOWER
I20260811 19:17:22.796521 7519 consensus_queue.cc:260] T 00000000000000000000000000000000 P 721dc63a2c7a49f0886a7016f935c642 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "721dc63a2c7a49f0886a7016f935c642" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44549 } }
I20260811 19:17:22.796900 7519 raft_consensus.cc:388] T 00000000000000000000000000000000 P 721dc63a2c7a49f0886a7016f935c642 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:17:22.797035 7519 raft_consensus.cc:482] T 00000000000000000000000000000000 P 721dc63a2c7a49f0886a7016f935c642 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:17:22.797233 7519 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 721dc63a2c7a49f0886a7016f935c642 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:17:22.799391 3318 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:17:22.801764 7519 raft_consensus.cc:504] T 00000000000000000000000000000000 P 721dc63a2c7a49f0886a7016f935c642 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "721dc63a2c7a49f0886a7016f935c642" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44549 } }
I20260811 19:17:22.802539 7519 leader_election.cc:302] T 00000000000000000000000000000000 P 721dc63a2c7a49f0886a7016f935c642 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 721dc63a2c7a49f0886a7016f935c642; no voters:
I20260811 19:17:22.803380 7519 leader_election.cc:288] T 00000000000000000000000000000000 P 721dc63a2c7a49f0886a7016f935c642 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:17:22.803509 7530 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 721dc63a2c7a49f0886a7016f935c642 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:17:22.804741 7530 raft_consensus.cc:686] T 00000000000000000000000000000000 P 721dc63a2c7a49f0886a7016f935c642 [term 1 LEADER]: Becoming Leader. State: Replica: 721dc63a2c7a49f0886a7016f935c642, State: Running, Role: LEADER
I20260811 19:17:22.805373 7530 consensus_queue.cc:237] T 00000000000000000000000000000000 P 721dc63a2c7a49f0886a7016f935c642 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "721dc63a2c7a49f0886a7016f935c642" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44549 } }
I20260811 19:17:22.806088 7519 sys_catalog.cc:564] T 00000000000000000000000000000000 P 721dc63a2c7a49f0886a7016f935c642 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:17:22.809085 7546 sys_catalog.cc:455] T 00000000000000000000000000000000 P 721dc63a2c7a49f0886a7016f935c642 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 721dc63a2c7a49f0886a7016f935c642. Latest consensus state: current_term: 1 leader_uuid: "721dc63a2c7a49f0886a7016f935c642" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "721dc63a2c7a49f0886a7016f935c642" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44549 } } }
I20260811 19:17:22.809746 7546 sys_catalog.cc:458] T 00000000000000000000000000000000 P 721dc63a2c7a49f0886a7016f935c642 [sys.catalog]: This master's current role is: LEADER
I20260811 19:17:22.809293 7542 sys_catalog.cc:455] T 00000000000000000000000000000000 P 721dc63a2c7a49f0886a7016f935c642 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "721dc63a2c7a49f0886a7016f935c642" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "721dc63a2c7a49f0886a7016f935c642" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44549 } } }
I20260811 19:17:22.809996 7542 sys_catalog.cc:458] T 00000000000000000000000000000000 P 721dc63a2c7a49f0886a7016f935c642 [sys.catalog]: This master's current role is: LEADER
I20260811 19:17:22.822299 7553 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:17:22.831310 7553 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:17:22.842476 7523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:22.805771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.70:47151 (local address 127.0.0.1:44549)
0811 19:17:22.806053 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:22.806061 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:22.817640 (+ 11579us) server_negotiation.cc:407] Connection header received
0811 19:17:22.818794 (+ 1154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:22.818800 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:22.818889 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:22.818968 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:22.820675 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:22.821643 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:22.823470 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:22.824220 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:22.824347 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:22.828706 (+ 4359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:22.828714 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:22.828721 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:17:22.828774 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:22.835584 (+ 6810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:22.838144 (+ 2560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:22.838152 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:22.838157 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:22.838423 (+ 266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:22.839206 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:22.839212 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:22.839222 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:22.839799 (+ 577us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:22.840159 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:22.841826 (+ 1667us) server_negotiation.cc:299] Negotiation successful
0811 19:17:22.842105 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141}
I20260811 19:17:22.844245 7553 catalog_manager.cc:1347] Generated new cluster ID: 8cf9dddfbc6b49ba8b8b86d51827b8f2
I20260811 19:17:22.844352 7553 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:17:22.873404 7553 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:17:22.875355 7553 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:17:22.886332 7553 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 721dc63a2c7a49f0886a7016f935c642: Generated new TSK 0
I20260811 19:17:22.887408 7553 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:17:22.900370 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 7393
I20260811 19:17:22.901876 7510 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:17:23.001787 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 7393
W20260811 19:17:23.032688 7356 connection.cc:535] client connection to 127.0.0.1:44549 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44549): Connection reset by peer (error 104)
2026-08-11T19:17:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2544 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786475454804547-3318-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:17:23 dist-test-slave-t7g5 krb5kdc[7588](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:17:23 dist-test-slave-t7g5 krb5kdc[7588](info): set up 2 sockets
Aug 11 19:17:23 dist-test-slave-t7g5 krb5kdc[7588](info): commencing operation
krb5kdc: starting...
W20260811 19:17:24.382814 3318 mini_kdc.cc:121] Time spent starting KDC: real 1.326s user 0.002s sys 0.068s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Aug 11 19:17:24 dist-test-slave-t7g5 krb5kdc[7588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475844, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:17:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:17:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786475454804547-3318-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786475454804547-3318-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 19:17:24.830506 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786475454804547-3318-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786475454804547-3318-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786475454804547-3318-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44725
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786475454804547-3318-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44725
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:17:25.257362 7610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:17:25.257776 7610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:17:25.257925 7610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:17:25.266927 7610 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:17:25.267187 7610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:17:25.267285 7610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:17:25.267380 7610 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:17:25.267445 7610 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:17:25.283162 7610 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786475454804547-3318-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786475454804547-3318-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44725
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786475454804547-3318-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44725
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786475454804547-3318-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:17:25.284792 7610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:17:25.286399 7610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:17:25.301398 7618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:17:25.301644 7625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:17:25.301591 7619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:17:25.302179 7610 server_base.cc:1034] running on GCE node
I20260811 19:17:25.303911 7610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:17:25.306264 7610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:17:25.307633 7610 hybrid_clock.cc:648] HybridClock initialized: now 1786475845307563 us; error 51 us; skew 500 ppm
I20260811 19:17:25.308198 7610 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:17:25 dist-test-slave-t7g5 krb5kdc[7588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475845, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:17:25.316428 7610 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 19:17:25.318499 7610 webserver.cc:460] Webserver started at http://127.0.0.1:46133/ using document root <none> and password file <none>
I20260811 19:17:25.319316 7610 fs_manager.cc:359] Metadata directory not provided
I20260811 19:17:25.319443 7610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:17:25.319878 7610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:17:25.323704 7610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786475454804547-3318-0/minicluster-data/master-0/data/instance:
uuid: "f58f7fdb338b4ead83d44c9792c53282"
format_stamp: "Formatted at 2026-08-11 19:17:25 on dist-test-slave-t7g5"
server_key: "f41e1fe3ea06148db93ce62495099acc"
server_key_iv: "e94950e2499ef7bf79490e82865c5d26"
server_key_version: "encryptionkey@0"
I20260811 19:17:25.324687 7610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786475454804547-3318-0/minicluster-data/master-0/wal/instance:
uuid: "f58f7fdb338b4ead83d44c9792c53282"
format_stamp: "Formatted at 2026-08-11 19:17:25 on dist-test-slave-t7g5"
server_key: "f41e1fe3ea06148db93ce62495099acc"
server_key_iv: "e94950e2499ef7bf79490e82865c5d26"
server_key_version: "encryptionkey@0"
I20260811 19:17:25.332402 7610 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260811 19:17:25.337568 7636 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:17:25.339696 7610 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260811 19:17:25.339931 7610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786475454804547-3318-0/minicluster-data/master-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786475454804547-3318-0/minicluster-data/master-0/wal
uuid: "f58f7fdb338b4ead83d44c9792c53282"
format_stamp: "Formatted at 2026-08-11 19:17:25 on dist-test-slave-t7g5"
server_key: "f41e1fe3ea06148db93ce62495099acc"
server_key_iv: "e94950e2499ef7bf79490e82865c5d26"
server_key_version: "encryptionkey@0"
I20260811 19:17:25.340189 7610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786475454804547-3318-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786475454804547-3318-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786475454804547-3318-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:17:25.374953 7610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:17:25.380396 7610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:17:25.380722 7610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:17:25.403661 7610 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44725
I20260811 19:17:25.403726 7722 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44725 every 8 connection(s)
I20260811 19:17:25.405721 7610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
I20260811 19:17:25.406313 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 7610
I20260811 19:17:25.406497 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786475454804547-3318-0/minicluster-data/master-0/wal/instance
I20260811 19:17:25.406961 3318 external_mini_cluster.cc:1428] Setting key de3435c9c02c3ea79316cc0ebf23b0e6
I20260811 19:17:25.412207 7723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:17:25.423377 7729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:25.409370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.70:39515 (local address 127.0.0.1:44725)
0811 19:17:25.410601 (+ 1231us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:25.410619 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:25.410676 (+ 57us) server_negotiation.cc:407] Connection header received
0811 19:17:25.412873 (+ 2197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:25.412909 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:25.413485 (+ 576us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:25.413987 (+ 502us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:25.414666 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:25.416223 (+ 1557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:25.417785 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:25.418723 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:25.418946 (+ 223us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:25.420558 (+ 1612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:25.420569 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:25.420588 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:17:25.421252 (+ 664us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:25.421349 (+ 97us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:17:25.421379 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:25.421557 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:25.421900 (+ 343us) server_negotiation.cc:299] Negotiation successful
0811 19:17:25.422110 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":874,"thread_start_us":509,"threads_started":1}
I20260811 19:17:25.427018 7723 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f58f7fdb338b4ead83d44c9792c53282: Bootstrap starting.
I20260811 19:17:25.428596 3318 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:17:25.430748 7723 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f58f7fdb338b4ead83d44c9792c53282: Neither blocks nor log segments found. Creating new log.
I20260811 19:17:25.431949 7723 log.cc:824] T 00000000000000000000000000000000 P f58f7fdb338b4ead83d44c9792c53282: Log is configured to *not* fsync() on all Append() calls
I20260811 19:17:25.436954 7723 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f58f7fdb338b4ead83d44c9792c53282: No bootstrap required, opened a new log
I20260811 19:17:25.443074 7729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:25.435038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.70:56597 (local address 127.0.0.1:44725)
0811 19:17:25.435314 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:25.435321 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:25.435626 (+ 305us) server_negotiation.cc:407] Connection header received
0811 19:17:25.435973 (+ 347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:25.435977 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:25.436181 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:25.436328 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:25.437106 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:25.438263 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:25.439855 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:25.440391 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:25.440534 (+ 143us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:25.442203 (+ 1669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:25.442210 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:25.442216 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:17:25.442243 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:25.442314 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:17:25.442339 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:25.442459 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:25.442692 (+ 233us) server_negotiation.cc:299] Negotiation successful
0811 19:17:25.442761 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260811 19:17:25.447009 7723 raft_consensus.cc:348] T 00000000000000000000000000000000 P f58f7fdb338b4ead83d44c9792c53282 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f58f7fdb338b4ead83d44c9792c53282" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44725 } }
I20260811 19:17:25.447489 7723 raft_consensus.cc:374] T 00000000000000000000000000000000 P f58f7fdb338b4ead83d44c9792c53282 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:17:25.447589 7723 raft_consensus.cc:729] T 00000000000000000000000000000000 P f58f7fdb338b4ead83d44c9792c53282 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f58f7fdb338b4ead83d44c9792c53282, State: Initialized, Role: FOLLOWER
I20260811 19:17:25.448279 7723 consensus_queue.cc:260] T 00000000000000000000000000000000 P f58f7fdb338b4ead83d44c9792c53282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f58f7fdb338b4ead83d44c9792c53282" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44725 } }
I20260811 19:17:25.448558 7723 raft_consensus.cc:388] T 00000000000000000000000000000000 P f58f7fdb338b4ead83d44c9792c53282 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:17:25.448701 7723 raft_consensus.cc:482] T 00000000000000000000000000000000 P f58f7fdb338b4ead83d44c9792c53282 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:17:25.448920 7723 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f58f7fdb338b4ead83d44c9792c53282 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:17:25.451424 7723 raft_consensus.cc:504] T 00000000000000000000000000000000 P f58f7fdb338b4ead83d44c9792c53282 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f58f7fdb338b4ead83d44c9792c53282" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44725 } }
I20260811 19:17:25.452035 7723 leader_election.cc:302] T 00000000000000000000000000000000 P f58f7fdb338b4ead83d44c9792c53282 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f58f7fdb338b4ead83d44c9792c53282; no voters:
I20260811 19:17:25.452703 7723 leader_election.cc:288] T 00000000000000000000000000000000 P f58f7fdb338b4ead83d44c9792c53282 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:17:25.452929 7741 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f58f7fdb338b4ead83d44c9792c53282 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:17:25.453632 7741 raft_consensus.cc:686] T 00000000000000000000000000000000 P f58f7fdb338b4ead83d44c9792c53282 [term 1 LEADER]: Becoming Leader. State: Replica: f58f7fdb338b4ead83d44c9792c53282, State: Running, Role: LEADER
I20260811 19:17:25.454694 7741 consensus_queue.cc:237] T 00000000000000000000000000000000 P f58f7fdb338b4ead83d44c9792c53282 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f58f7fdb338b4ead83d44c9792c53282" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44725 } }
I20260811 19:17:25.455052 7723 sys_catalog.cc:564] T 00000000000000000000000000000000 P f58f7fdb338b4ead83d44c9792c53282 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:17:25.457891 7744 sys_catalog.cc:455] T 00000000000000000000000000000000 P f58f7fdb338b4ead83d44c9792c53282 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f58f7fdb338b4ead83d44c9792c53282. Latest consensus state: current_term: 1 leader_uuid: "f58f7fdb338b4ead83d44c9792c53282" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f58f7fdb338b4ead83d44c9792c53282" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44725 } } }
I20260811 19:17:25.458451 7744 sys_catalog.cc:458] T 00000000000000000000000000000000 P f58f7fdb338b4ead83d44c9792c53282 [sys.catalog]: This master's current role is: LEADER
I20260811 19:17:25.459095 7743 sys_catalog.cc:455] T 00000000000000000000000000000000 P f58f7fdb338b4ead83d44c9792c53282 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f58f7fdb338b4ead83d44c9792c53282" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f58f7fdb338b4ead83d44c9792c53282" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44725 } } }
I20260811 19:17:25.459458 7743 sys_catalog.cc:458] T 00000000000000000000000000000000 P f58f7fdb338b4ead83d44c9792c53282 [sys.catalog]: This master's current role is: LEADER
I20260811 19:17:25.459460 7753 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:17:25.468004 7753 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:17:25.479214 7753 catalog_manager.cc:1347] Generated new cluster ID: 3a26bccc30174d98ae14bd0cfc6ff0cb
I20260811 19:17:25.479373 7753 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:17:25.496061 7753 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:17:25.498060 7753 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:17:25.512135 7753 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f58f7fdb338b4ead83d44c9792c53282: Generated new TSK 0
I20260811 19:17:25.513437 7753 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:17:25.525130 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 7610
W20260811 19:17:25.526597 7713 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.70:39515
W20260811 19:17:25.527419 3318 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 7610: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 19:17:25.527559 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 7610
W20260811 19:17:25.557794 7584 connection.cc:535] client connection to 127.0.0.1:44725 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44725): Connection reset by peer (error 104)
2026-08-11T19:17:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2523 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786475454804547-3318-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:17:25 dist-test-slave-t7g5 krb5kdc[7789](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:17:25 dist-test-slave-t7g5 krb5kdc[7789](info): set up 2 sockets
Aug 11 19:17:25 dist-test-slave-t7g5 krb5kdc[7789](info): commencing operation
krb5kdc: starting...
W20260811 19:17:26.896004 3318 mini_kdc.cc:121] Time spent starting KDC: real 1.315s user 0.001s sys 0.068s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Aug 11 19:17:27 dist-test-slave-t7g5 krb5kdc[7789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475847, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:17:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:17:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786475454804547-3318-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786475454804547-3318-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 19:17:27.353698 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786475454804547-3318-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786475454804547-3318-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786475454804547-3318-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45965
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786475454804547-3318-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45965
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:17:27.794170 7807 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:17:27.794550 7807 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:17:27.794670 7807 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:17:27.806245 7807 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:17:27.806432 7807 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:17:27.806488 7807 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:17:27.806571 7807 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:17:27.806715 7807 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:17:27.818775 7807 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786475454804547-3318-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786475454804547-3318-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45965
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786475454804547-3318-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45965
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786475454804547-3318-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:17:27.820359 7807 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:17:27.822063 7807 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:17:27.837574 7822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:17:27.837522 7819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:17:27.837615 7818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:17:27.838510 7807 server_base.cc:1034] running on GCE node
I20260811 19:17:27.839520 7807 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:17:27.841486 7807 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:17:27.842909 7807 hybrid_clock.cc:648] HybridClock initialized: now 1786475847842833 us; error 66 us; skew 500 ppm
I20260811 19:17:27.843536 7807 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:17:27 dist-test-slave-t7g5 krb5kdc[7789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475847, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:17:27.848279 7807 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 19:17:27.850492 7807 webserver.cc:460] Webserver started at http://127.0.0.1:33865/ using document root <none> and password file <none>
I20260811 19:17:27.851401 7807 fs_manager.cc:359] Metadata directory not provided
I20260811 19:17:27.851548 7807 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:17:27.851886 7807 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:17:27.855615 7807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786475454804547-3318-0/minicluster-data/master-0/data/instance:
uuid: "1dd833763e254981817638bd1733d785"
format_stamp: "Formatted at 2026-08-11 19:17:27 on dist-test-slave-t7g5"
server_key: "65d09235f5c25dfb166ad0a9663e57f1"
server_key_iv: "1d28792e25fab023c3cedbae9cbf2c90"
server_key_version: "encryptionkey@0"
I20260811 19:17:27.856565 7807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786475454804547-3318-0/minicluster-data/master-0/wal/instance:
uuid: "1dd833763e254981817638bd1733d785"
format_stamp: "Formatted at 2026-08-11 19:17:27 on dist-test-slave-t7g5"
server_key: "65d09235f5c25dfb166ad0a9663e57f1"
server_key_iv: "1d28792e25fab023c3cedbae9cbf2c90"
server_key_version: "encryptionkey@0"
I20260811 19:17:27.863776 7807 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.008s sys 0.000s
I20260811 19:17:27.869315 7837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:17:27.871881 7807 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260811 19:17:27.872186 7807 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786475454804547-3318-0/minicluster-data/master-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786475454804547-3318-0/minicluster-data/master-0/wal
uuid: "1dd833763e254981817638bd1733d785"
format_stamp: "Formatted at 2026-08-11 19:17:27 on dist-test-slave-t7g5"
server_key: "65d09235f5c25dfb166ad0a9663e57f1"
server_key_iv: "1d28792e25fab023c3cedbae9cbf2c90"
server_key_version: "encryptionkey@0"
I20260811 19:17:27.872498 7807 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786475454804547-3318-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786475454804547-3318-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786475454804547-3318-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:17:27.879809 7807 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:17:27.886530 7807 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:17:27.887039 7807 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:17:27.910488 7807 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45965
I20260811 19:17:27.910537 7930 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45965 every 8 connection(s)
I20260811 19:17:27.912207 7807 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
I20260811 19:17:27.916991 7932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:17:27.917699 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 7807
I20260811 19:17:27.917913 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786475454804547-3318-0/minicluster-data/master-0/wal/instance
I20260811 19:17:27.918502 3318 external_mini_cluster.cc:1428] Setting key 4ffab81fdfe877d13c40fa834c147ddb
I20260811 19:17:27.930512 7936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:27.920845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.70:34639 (local address 127.0.0.1:45965)
0811 19:17:27.921799 (+ 954us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:27.921813 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:27.921868 (+ 55us) server_negotiation.cc:407] Connection header received
0811 19:17:27.924859 (+ 2991us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:27.924918 (+ 59us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:27.925870 (+ 952us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:27.926413 (+ 543us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:27.927880 (+ 1467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:27.927891 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:27.927921 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:17:27.927957 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:27.928074 (+ 117us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:17:27.928110 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:27.928419 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:27.928606 (+ 187us) server_negotiation.cc:299] Negotiation successful
0811 19:17:27.928935 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":648,"thread_start_us":415,"threads_started":1}
I20260811 19:17:27.933876 7932 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1dd833763e254981817638bd1733d785: Bootstrap starting.
I20260811 19:17:27.935906 3318 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:17:27.938344 7932 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1dd833763e254981817638bd1733d785: Neither blocks nor log segments found. Creating new log.
I20260811 19:17:27.940598 7932 log.cc:824] T 00000000000000000000000000000000 P 1dd833763e254981817638bd1733d785: Log is configured to *not* fsync() on all Append() calls
I20260811 19:17:27.946161 7932 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1dd833763e254981817638bd1733d785: No bootstrap required, opened a new log
I20260811 19:17:27.946691 7936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:27.942903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.70:47497 (local address 127.0.0.1:45965)
0811 19:17:27.943158 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:27.943166 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:27.943636 (+ 470us) server_negotiation.cc:407] Connection header received
0811 19:17:27.944001 (+ 365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:27.944007 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:27.944096 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:27.944241 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:27.945607 (+ 1366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:27.945614 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:27.945618 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:17:27.945648 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:27.945776 (+ 128us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:17:27.945797 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:27.945930 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:27.946213 (+ 283us) server_negotiation.cc:299] Negotiation successful
0811 19:17:27.946333 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 19:17:27.955848 7932 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1dd833763e254981817638bd1733d785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dd833763e254981817638bd1733d785" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45965 } }
I20260811 19:17:27.956586 7932 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1dd833763e254981817638bd1733d785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:17:27.956719 7932 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1dd833763e254981817638bd1733d785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dd833763e254981817638bd1733d785, State: Initialized, Role: FOLLOWER
I20260811 19:17:27.957485 7932 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1dd833763e254981817638bd1733d785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dd833763e254981817638bd1733d785" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45965 } }
I20260811 19:17:27.957868 7932 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1dd833763e254981817638bd1733d785 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:17:27.958029 7932 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1dd833763e254981817638bd1733d785 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:17:27.958220 7932 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1dd833763e254981817638bd1733d785 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:17:27.960835 7932 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1dd833763e254981817638bd1733d785 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dd833763e254981817638bd1733d785" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45965 } }
I20260811 19:17:27.961601 7932 leader_election.cc:302] T 00000000000000000000000000000000 P 1dd833763e254981817638bd1733d785 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1dd833763e254981817638bd1733d785; no voters:
I20260811 19:17:27.962476 7932 leader_election.cc:288] T 00000000000000000000000000000000 P 1dd833763e254981817638bd1733d785 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:17:27.962603 7947 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1dd833763e254981817638bd1733d785 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:17:27.963301 7947 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1dd833763e254981817638bd1733d785 [term 1 LEADER]: Becoming Leader. State: Replica: 1dd833763e254981817638bd1733d785, State: Running, Role: LEADER
I20260811 19:17:27.964067 7947 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1dd833763e254981817638bd1733d785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dd833763e254981817638bd1733d785" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45965 } }
I20260811 19:17:27.965782 7932 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1dd833763e254981817638bd1733d785 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:17:27.968987 7948 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dd833763e254981817638bd1733d785 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1dd833763e254981817638bd1733d785" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dd833763e254981817638bd1733d785" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45965 } } }
I20260811 19:17:27.970463 7948 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dd833763e254981817638bd1733d785 [sys.catalog]: This master's current role is: LEADER
I20260811 19:17:27.968469 7949 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dd833763e254981817638bd1733d785 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1dd833763e254981817638bd1733d785. Latest consensus state: current_term: 1 leader_uuid: "1dd833763e254981817638bd1733d785" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dd833763e254981817638bd1733d785" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45965 } } }
I20260811 19:17:27.972219 7949 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dd833763e254981817638bd1733d785 [sys.catalog]: This master's current role is: LEADER
I20260811 19:17:27.973058 7959 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:17:27.978745 7959 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:17:27.993000 7959 catalog_manager.cc:1347] Generated new cluster ID: e8f11c560733450db4546aeb399eae06
I20260811 19:17:27.993155 7959 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:17:28.012574 7959 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:17:28.013114 7959 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:17:28.021412 7959 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1dd833763e254981817638bd1733d785: Generated new TSK 0
I20260811 19:17:28.022517 7959 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:17:28.034336 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 7807
W20260811 19:17:28.035936 7921 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.70:34639
W20260811 19:17:28.036783 3318 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 7807: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 19:17:28.036976 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 7807
2026-08-11T19:17:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2509 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786475454804547-3318-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:17:28 dist-test-slave-t7g5 krb5kdc[7989](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:17:28 dist-test-slave-t7g5 krb5kdc[7989](info): set up 2 sockets
Aug 11 19:17:28 dist-test-slave-t7g5 krb5kdc[7989](info): commencing operation
krb5kdc: starting...
W20260811 19:17:29.415743 3318 mini_kdc.cc:121] Time spent starting KDC: real 1.326s user 0.002s sys 0.070s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Aug 11 19:17:29 dist-test-slave-t7g5 krb5kdc[7989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475849, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:17:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:17:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.61.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab.
Entry for principal kudu/127.3.61.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab.
I20260811 19:17:29.843321 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786475454804547-3318-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786475454804547-3318-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786475454804547-3318-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:32989
--webserver_interface=127.3.61.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:36145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.61.190:32989
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:17:30.272751 8013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:17:30.273130 8013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:17:30.273272 8013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:17:30.282753 8013 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:17:30.282931 8013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:17:30.283006 8013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:17:30.283119 8013 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:17:30.283195 8013 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:17:30.294384 8013 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:36145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786475454804547-3318-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786475454804547-3318-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.61.190:32989
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:32989
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.61.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786475454804547-3318-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:17:30.296187 8013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:17:30.298296 8013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:17:30.312628 8019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:17:30.312870 8023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:17:30.313012 8020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:17:30.314772 8013 server_base.cc:1034] running on GCE node
I20260811 19:17:30.315856 8013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:17:30.318128 8013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:17:30.319463 8013 hybrid_clock.cc:648] HybridClock initialized: now 1786475850319400 us; error 59 us; skew 500 ppm
I20260811 19:17:30.319942 8013 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:17:30 dist-test-slave-t7g5 krb5kdc[7989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475850, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:17:30.326857 8013 init.cc:377] Logged in from keytab as kudu/127.3.61.190@KRBTEST.COM (short username kudu)
I20260811 19:17:30.329005 8013 webserver.cc:460] Webserver started at http://127.3.61.190:32937/ using document root <none> and password file <none>
I20260811 19:17:30.329887 8013 fs_manager.cc:359] Metadata directory not provided
I20260811 19:17:30.330029 8013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:17:30.330363 8013 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:17:30.333793 8013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786475454804547-3318-0/minicluster-data/master-0/data/instance:
uuid: "13ec879a9f064244a7287978c7f9a4c5"
format_stamp: "Formatted at 2026-08-11 19:17:30 on dist-test-slave-t7g5"
server_key: "9611dabca951cef534ce919af09cf7e4"
server_key_iv: "0efed6728b61af6434ff1dcc61eb6c35"
server_key_version: "encryptionkey@0"
I20260811 19:17:30.334712 8013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786475454804547-3318-0/minicluster-data/master-0/wal/instance:
uuid: "13ec879a9f064244a7287978c7f9a4c5"
format_stamp: "Formatted at 2026-08-11 19:17:30 on dist-test-slave-t7g5"
server_key: "9611dabca951cef534ce919af09cf7e4"
server_key_iv: "0efed6728b61af6434ff1dcc61eb6c35"
server_key_version: "encryptionkey@0"
I20260811 19:17:30.341929 8013 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260811 19:17:30.347703 8038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:17:30.349920 8013 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.003s
I20260811 19:17:30.350183 8013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786475454804547-3318-0/minicluster-data/master-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786475454804547-3318-0/minicluster-data/master-0/wal
uuid: "13ec879a9f064244a7287978c7f9a4c5"
format_stamp: "Formatted at 2026-08-11 19:17:30 on dist-test-slave-t7g5"
server_key: "9611dabca951cef534ce919af09cf7e4"
server_key_iv: "0efed6728b61af6434ff1dcc61eb6c35"
server_key_version: "encryptionkey@0"
I20260811 19:17:30.350433 8013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786475454804547-3318-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786475454804547-3318-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786475454804547-3318-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:17:30.371464 8013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:17:30.376406 8013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:17:30.376724 8013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:17:30.400173 8013 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.190:32989
I20260811 19:17:30.400208 8141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.190:32989 every 8 connection(s)
I20260811 19:17:30.401937 8013 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
I20260811 19:17:30.407460 8142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:17:30.409129 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 8013
I20260811 19:17:30.409380 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786475454804547-3318-0/minicluster-data/master-0/wal/instance
I20260811 19:17:30.410040 3318 external_mini_cluster.cc:1428] Setting key bc3bf096837be4df1ee4bbb0dab6ddce
I20260811 19:17:30.423821 8142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13ec879a9f064244a7287978c7f9a4c5: Bootstrap starting.
I20260811 19:17:30.427588 8142 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 13ec879a9f064244a7287978c7f9a4c5: Neither blocks nor log segments found. Creating new log.
Aug 11 19:17:30 dist-test-slave-t7g5 krb5kdc[7989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475849, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:17:30.429615 8142 log.cc:824] T 00000000000000000000000000000000 P 13ec879a9f064244a7287978c7f9a4c5: Log is configured to *not* fsync() on all Append() calls
I20260811 19:17:30.434022 8142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13ec879a9f064244a7287978c7f9a4c5: No bootstrap required, opened a new log
I20260811 19:17:30.441202 8147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:30.412188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51063 (local address 127.3.61.190:32989)
0811 19:17:30.413359 (+ 1171us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:30.413373 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:30.413555 (+ 182us) server_negotiation.cc:407] Connection header received
0811 19:17:30.416152 (+ 2597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:30.416191 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:30.416752 (+ 561us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:30.417226 (+ 474us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:30.419492 (+ 2266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:30.421242 (+ 1750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:30.423084 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:30.423972 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:30.424203 (+ 231us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:30.430187 (+ 5984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:30.430199 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:30.430223 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:17:30.430302 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:30.434090 (+ 3788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:30.435263 (+ 1173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:30.435271 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:30.435280 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:30.435617 (+ 337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:30.436687 (+ 1070us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:30.436692 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:30.436695 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:30.437526 (+ 831us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:30.438024 (+ 498us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:30.438635 (+ 611us) server_negotiation.cc:299] Negotiation successful
0811 19:17:30.439211 (+ 576us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":857,"thread_start_us":451,"threads_started":1}
I20260811 19:17:30.444533 8142 raft_consensus.cc:348] T 00000000000000000000000000000000 P 13ec879a9f064244a7287978c7f9a4c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13ec879a9f064244a7287978c7f9a4c5" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 32989 } }
I20260811 19:17:30.445102 8142 raft_consensus.cc:374] T 00000000000000000000000000000000 P 13ec879a9f064244a7287978c7f9a4c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:17:30.445222 8142 raft_consensus.cc:729] T 00000000000000000000000000000000 P 13ec879a9f064244a7287978c7f9a4c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13ec879a9f064244a7287978c7f9a4c5, State: Initialized, Role: FOLLOWER
I20260811 19:17:30.446048 8142 consensus_queue.cc:260] T 00000000000000000000000000000000 P 13ec879a9f064244a7287978c7f9a4c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13ec879a9f064244a7287978c7f9a4c5" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 32989 } }
I20260811 19:17:30.446381 8142 raft_consensus.cc:388] T 00000000000000000000000000000000 P 13ec879a9f064244a7287978c7f9a4c5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:17:30.446542 8142 raft_consensus.cc:482] T 00000000000000000000000000000000 P 13ec879a9f064244a7287978c7f9a4c5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:17:30.446780 8142 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 13ec879a9f064244a7287978c7f9a4c5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:17:30.447968 3318 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:17:30.450184 8142 raft_consensus.cc:504] T 00000000000000000000000000000000 P 13ec879a9f064244a7287978c7f9a4c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13ec879a9f064244a7287978c7f9a4c5" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 32989 } }
I20260811 19:17:30.450987 8142 leader_election.cc:302] T 00000000000000000000000000000000 P 13ec879a9f064244a7287978c7f9a4c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 13ec879a9f064244a7287978c7f9a4c5; no voters:
I20260811 19:17:30.451808 8142 leader_election.cc:288] T 00000000000000000000000000000000 P 13ec879a9f064244a7287978c7f9a4c5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:17:30.451938 8155 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 13ec879a9f064244a7287978c7f9a4c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:17:30.452737 8155 raft_consensus.cc:686] T 00000000000000000000000000000000 P 13ec879a9f064244a7287978c7f9a4c5 [term 1 LEADER]: Becoming Leader. State: Replica: 13ec879a9f064244a7287978c7f9a4c5, State: Running, Role: LEADER
I20260811 19:17:30.453714 8155 consensus_queue.cc:237] T 00000000000000000000000000000000 P 13ec879a9f064244a7287978c7f9a4c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13ec879a9f064244a7287978c7f9a4c5" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 32989 } }
I20260811 19:17:30.455673 8142 sys_catalog.cc:564] T 00000000000000000000000000000000 P 13ec879a9f064244a7287978c7f9a4c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:17:30.457816 8161 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13ec879a9f064244a7287978c7f9a4c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "13ec879a9f064244a7287978c7f9a4c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13ec879a9f064244a7287978c7f9a4c5" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 32989 } } }
I20260811 19:17:30.457957 8160 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13ec879a9f064244a7287978c7f9a4c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 13ec879a9f064244a7287978c7f9a4c5. Latest consensus state: current_term: 1 leader_uuid: "13ec879a9f064244a7287978c7f9a4c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13ec879a9f064244a7287978c7f9a4c5" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 32989 } } }
I20260811 19:17:30.458405 8160 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13ec879a9f064244a7287978c7f9a4c5 [sys.catalog]: This master's current role is: LEADER
I20260811 19:17:30.458406 8161 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13ec879a9f064244a7287978c7f9a4c5 [sys.catalog]: This master's current role is: LEADER
I20260811 19:17:30.479076 8147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:30.454093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41441 (local address 127.3.61.190:32989)
0811 19:17:30.454381 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:30.454389 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:30.455377 (+ 988us) server_negotiation.cc:407] Connection header received
0811 19:17:30.455660 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:30.455666 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:30.455764 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:30.455846 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:30.458070 (+ 2224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:30.459464 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:30.462288 (+ 2824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:30.463135 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:30.463300 (+ 165us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:30.468135 (+ 4835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:30.468145 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:30.468151 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:17:30.468235 (+ 84us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:30.472662 (+ 4427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:30.474659 (+ 1997us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:30.474669 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:30.474674 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:30.474974 (+ 300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:30.475724 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:30.475730 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:30.475734 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:30.476252 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:30.476580 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:30.478311 (+ 1731us) server_negotiation.cc:299] Negotiation successful
0811 19:17:30.478616 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260811 19:17:30.481997 8173 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:17:30.488130 8173 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:17:30.499282 8173 catalog_manager.cc:1347] Generated new cluster ID: 40be2819cb434d49a61044742e206e71
I20260811 19:17:30.499437 8173 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:17:30.534315 8173 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:17:30.536990 8173 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:17:30.548324 8173 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 13ec879a9f064244a7287978c7f9a4c5: Generated new TSK 0
I20260811 19:17:30.549710 8173 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:17:30.562788 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 8013
I20260811 19:17:30.565054 8129 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:17:30.662640 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 8013
W20260811 19:17:30.693940 7983 connection.cc:535] client connection to 127.3.61.190:32989 recv error: Network error: failed to read from TLS socket (remote: 127.3.61.190:32989): Connection reset by peer (error 104)
2026-08-11T19:17:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2624 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786475454804547-3318-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:17:30 dist-test-slave-t7g5 krb5kdc[8204](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:17:30 dist-test-slave-t7g5 krb5kdc[8204](info): set up 2 sockets
Aug 11 19:17:30 dist-test-slave-t7g5 krb5kdc[8204](info): commencing operation
krb5kdc: starting...
W20260811 19:17:32.050072 3318 mini_kdc.cc:121] Time spent starting KDC: real 1.335s user 0.006s sys 0.069s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Aug 11 19:17:32 dist-test-slave-t7g5 krb5kdc[8204](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:17:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:17:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.61.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab.
Entry for principal kudu/127.3.61.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab.
I20260811 19:17:32.487589 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786475454804547-3318-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786475454804547-3318-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786475454804547-3318-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:38317
--webserver_interface=127.3.61.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:35525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.61.190:38317
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:17:32.902523 8231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:17:32.902907 8231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:17:32.903046 8231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:17:32.912402 8231 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:17:32.912631 8231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:17:32.912710 8231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:17:32.912827 8231 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:17:32.912892 8231 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:17:32.925201 8231 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:35525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786475454804547-3318-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786475454804547-3318-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.61.190:38317
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:38317
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.61.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786475454804547-3318-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:17:32.926757 8231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:17:32.928224 8231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:17:32.942395 8247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:17:32.942479 8241 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:17:32.942432 8245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:17:32.944013 8231 server_base.cc:1034] running on GCE node
I20260811 19:17:32.945564 8231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:17:32.947453 8231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:17:32.948838 8231 hybrid_clock.cc:648] HybridClock initialized: now 1786475852948746 us; error 50 us; skew 500 ppm
I20260811 19:17:32.949496 8231 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:17:32 dist-test-slave-t7g5 krb5kdc[8204](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475852, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:17:32.954257 8231 init.cc:377] Logged in from keytab as kudu/127.3.61.190@KRBTEST.COM (short username kudu)
I20260811 19:17:32.956558 8231 webserver.cc:460] Webserver started at http://127.3.61.190:34521/ using document root <none> and password file <none>
I20260811 19:17:32.957441 8231 fs_manager.cc:359] Metadata directory not provided
I20260811 19:17:32.957603 8231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:17:32.958009 8231 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:17:32.961766 8231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786475454804547-3318-0/minicluster-data/master-0/data/instance:
uuid: "92173aa9eaeb4cb6bdc4b00f8623666e"
format_stamp: "Formatted at 2026-08-11 19:17:32 on dist-test-slave-t7g5"
server_key: "fff3934e852843d932b808c505a790e7"
server_key_iv: "01284a57e06d9349462152b07d958195"
server_key_version: "encryptionkey@0"
I20260811 19:17:32.962774 8231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786475454804547-3318-0/minicluster-data/master-0/wal/instance:
uuid: "92173aa9eaeb4cb6bdc4b00f8623666e"
format_stamp: "Formatted at 2026-08-11 19:17:32 on dist-test-slave-t7g5"
server_key: "fff3934e852843d932b808c505a790e7"
server_key_iv: "01284a57e06d9349462152b07d958195"
server_key_version: "encryptionkey@0"
I20260811 19:17:32.970121 8231 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260811 19:17:32.975956 8265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:17:32.977849 8231 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.004s
I20260811 19:17:32.978089 8231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786475454804547-3318-0/minicluster-data/master-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786475454804547-3318-0/minicluster-data/master-0/wal
uuid: "92173aa9eaeb4cb6bdc4b00f8623666e"
format_stamp: "Formatted at 2026-08-11 19:17:32 on dist-test-slave-t7g5"
server_key: "fff3934e852843d932b808c505a790e7"
server_key_iv: "01284a57e06d9349462152b07d958195"
server_key_version: "encryptionkey@0"
I20260811 19:17:32.978372 8231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786475454804547-3318-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786475454804547-3318-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786475454804547-3318-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:17:33.029023 8231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:17:33.034988 8231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:17:33.035485 8231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:17:33.058943 8231 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.190:38317
I20260811 19:17:33.059091 8378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.190:38317 every 8 connection(s)
I20260811 19:17:33.060491 8231 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
I20260811 19:17:33.066301 8379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:17:33.068955 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 8231
I20260811 19:17:33.069218 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786475454804547-3318-0/minicluster-data/master-0/wal/instance
I20260811 19:17:33.070016 3318 external_mini_cluster.cc:1428] Setting key d5d9b964af0269f3189222ef2f8dbacd
I20260811 19:17:33.081698 8379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92173aa9eaeb4cb6bdc4b00f8623666e: Bootstrap starting.
I20260811 19:17:33.086925 8379 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 92173aa9eaeb4cb6bdc4b00f8623666e: Neither blocks nor log segments found. Creating new log.
I20260811 19:17:33.088711 8379 log.cc:824] T 00000000000000000000000000000000 P 92173aa9eaeb4cb6bdc4b00f8623666e: Log is configured to *not* fsync() on all Append() calls
I20260811 19:17:33.089109 8387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:33.073262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42195 (local address 127.3.61.190:38317)
0811 19:17:33.074337 (+ 1075us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:33.074359 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:33.074425 (+ 66us) server_negotiation.cc:407] Connection header received
0811 19:17:33.076665 (+ 2240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:33.076699 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:33.077225 (+ 526us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:33.077747 (+ 522us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:33.078563 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:33.080558 (+ 1995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:33.082294 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:33.083119 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:33.083364 (+ 245us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:33.085381 (+ 2017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:33.085392 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:33.085415 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:17:33.086296 (+ 881us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:33.086424 (+ 128us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:17:33.086463 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:33.086655 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:33.087270 (+ 615us) server_negotiation.cc:299] Negotiation successful
0811 19:17:33.087516 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":754,"thread_start_us":453,"threads_started":1}
I20260811 19:17:33.092799 8379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92173aa9eaeb4cb6bdc4b00f8623666e: No bootstrap required, opened a new log
I20260811 19:17:33.095191 3318 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:17:33.105295 8379 raft_consensus.cc:348] T 00000000000000000000000000000000 P 92173aa9eaeb4cb6bdc4b00f8623666e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92173aa9eaeb4cb6bdc4b00f8623666e" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 38317 } }
I20260811 19:17:33.106276 8379 raft_consensus.cc:374] T 00000000000000000000000000000000 P 92173aa9eaeb4cb6bdc4b00f8623666e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:17:33.106452 8379 raft_consensus.cc:729] T 00000000000000000000000000000000 P 92173aa9eaeb4cb6bdc4b00f8623666e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92173aa9eaeb4cb6bdc4b00f8623666e, State: Initialized, Role: FOLLOWER
I20260811 19:17:33.107286 8379 consensus_queue.cc:260] T 00000000000000000000000000000000 P 92173aa9eaeb4cb6bdc4b00f8623666e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92173aa9eaeb4cb6bdc4b00f8623666e" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 38317 } }
I20260811 19:17:33.107600 8379 raft_consensus.cc:388] T 00000000000000000000000000000000 P 92173aa9eaeb4cb6bdc4b00f8623666e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:17:33.107771 8379 raft_consensus.cc:482] T 00000000000000000000000000000000 P 92173aa9eaeb4cb6bdc4b00f8623666e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:17:33.107997 8379 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 92173aa9eaeb4cb6bdc4b00f8623666e [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:17:33.110478 8387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:33.101286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43059 (local address 127.3.61.190:38317)
0811 19:17:33.101494 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:33.101498 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:33.101997 (+ 499us) server_negotiation.cc:407] Connection header received
0811 19:17:33.102127 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:33.102132 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:33.102259 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:33.102420 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:33.103554 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:33.104397 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:33.107020 (+ 2623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:33.107632 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:33.107805 (+ 173us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:33.109574 (+ 1769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:33.109580 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:33.109584 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:17:33.109609 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:33.109708 (+ 99us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:17:33.109734 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:33.109872 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:33.110091 (+ 219us) server_negotiation.cc:299] Negotiation successful
0811 19:17:33.110161 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 19:17:33.111493 8379 raft_consensus.cc:504] T 00000000000000000000000000000000 P 92173aa9eaeb4cb6bdc4b00f8623666e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92173aa9eaeb4cb6bdc4b00f8623666e" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 38317 } }
I20260811 19:17:33.112368 8379 leader_election.cc:302] T 00000000000000000000000000000000 P 92173aa9eaeb4cb6bdc4b00f8623666e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 92173aa9eaeb4cb6bdc4b00f8623666e; no voters:
I20260811 19:17:33.113209 8379 leader_election.cc:288] T 00000000000000000000000000000000 P 92173aa9eaeb4cb6bdc4b00f8623666e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:17:33.113365 8407 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 92173aa9eaeb4cb6bdc4b00f8623666e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:17:33.114217 8407 raft_consensus.cc:686] T 00000000000000000000000000000000 P 92173aa9eaeb4cb6bdc4b00f8623666e [term 1 LEADER]: Becoming Leader. State: Replica: 92173aa9eaeb4cb6bdc4b00f8623666e, State: Running, Role: LEADER
I20260811 19:17:33.114980 8407 consensus_queue.cc:237] T 00000000000000000000000000000000 P 92173aa9eaeb4cb6bdc4b00f8623666e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92173aa9eaeb4cb6bdc4b00f8623666e" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 38317 } }
I20260811 19:17:33.115617 8379 sys_catalog.cc:564] T 00000000000000000000000000000000 P 92173aa9eaeb4cb6bdc4b00f8623666e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:17:33.118724 8411 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92173aa9eaeb4cb6bdc4b00f8623666e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "92173aa9eaeb4cb6bdc4b00f8623666e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92173aa9eaeb4cb6bdc4b00f8623666e" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 38317 } } }
I20260811 19:17:33.118697 8412 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92173aa9eaeb4cb6bdc4b00f8623666e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92173aa9eaeb4cb6bdc4b00f8623666e. Latest consensus state: current_term: 1 leader_uuid: "92173aa9eaeb4cb6bdc4b00f8623666e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92173aa9eaeb4cb6bdc4b00f8623666e" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 38317 } } }
I20260811 19:17:33.119262 8411 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92173aa9eaeb4cb6bdc4b00f8623666e [sys.catalog]: This master's current role is: LEADER
I20260811 19:17:33.119257 8412 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92173aa9eaeb4cb6bdc4b00f8623666e [sys.catalog]: This master's current role is: LEADER
I20260811 19:17:33.121333 8417 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:17:33.128887 8417 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:17:33.148662 8417 catalog_manager.cc:1347] Generated new cluster ID: 5bd75d8e8577478589d711b2b33d4fca
I20260811 19:17:33.148854 8417 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:17:33.164582 8417 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:17:33.167155 8417 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:17:33.180639 8417 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 92173aa9eaeb4cb6bdc4b00f8623666e: Generated new TSK 0
I20260811 19:17:33.181819 8417 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:17:33.200163 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 8231
W20260811 19:17:33.201578 8371 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:42195
W20260811 19:17:33.202471 3318 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 8231: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 19:17:33.202618 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 8231
W20260811 19:17:33.238582 8198 connection.cc:535] client connection to 127.3.61.190:38317 recv error: Network error: failed to read from TLS socket (remote: 127.3.61.190:38317): Connection reset by peer (error 104)
2026-08-11T19:17:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2545 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786475454804547-3318-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:17:33 dist-test-slave-t7g5 krb5kdc[8463](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:17:33 dist-test-slave-t7g5 krb5kdc[8463](info): set up 2 sockets
Aug 11 19:17:33 dist-test-slave-t7g5 krb5kdc[8463](info): commencing operation
krb5kdc: starting...
W20260811 19:17:34.588289 3318 mini_kdc.cc:121] Time spent starting KDC: real 1.329s user 0.001s sys 0.073s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Aug 11 19:17:34 dist-test-slave-t7g5 krb5kdc[8463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:17:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:17:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.61.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab.
Entry for principal kudu/127.3.61.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab.
I20260811 19:17:35.019687 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786475454804547-3318-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786475454804547-3318-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786475454804547-3318-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:34583
--webserver_interface=127.3.61.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:35867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.61.190:34583
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:17:35.443415 8488 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:17:35.443774 8488 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:17:35.443908 8488 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:17:35.453754 8488 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:17:35.453935 8488 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:17:35.454020 8488 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:17:35.454118 8488 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:17:35.454177 8488 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:17:35.465250 8488 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:35867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786475454804547-3318-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786475454804547-3318-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.61.190:34583
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:34583
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.61.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786475454804547-3318-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:17:35.466827 8488 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:17:35.468303 8488 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:17:35.482398 8502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:17:35.482465 8504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:17:35.482455 8499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:17:35.483372 8488 server_base.cc:1034] running on GCE node
I20260811 19:17:35.484325 8488 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:17:35.486466 8488 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:17:35.487895 8488 hybrid_clock.cc:648] HybridClock initialized: now 1786475855487783 us; error 88 us; skew 500 ppm
I20260811 19:17:35.488435 8488 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:17:35 dist-test-slave-t7g5 krb5kdc[8463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475855, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:17:35.500869 8488 init.cc:377] Logged in from keytab as kudu/127.3.61.190@KRBTEST.COM (short username kudu)
I20260811 19:17:35.502913 8488 webserver.cc:460] Webserver started at http://127.3.61.190:45119/ using document root <none> and password file <none>
I20260811 19:17:35.503767 8488 fs_manager.cc:359] Metadata directory not provided
I20260811 19:17:35.503919 8488 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:17:35.504304 8488 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:17:35.507800 8488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786475454804547-3318-0/minicluster-data/master-0/data/instance:
uuid: "89f8f76db6c845489ec58e7f6256fe8d"
format_stamp: "Formatted at 2026-08-11 19:17:35 on dist-test-slave-t7g5"
server_key: "d3b0570cf59a808381c80bbd615e4954"
server_key_iv: "503b76bb3f5b2e1cbb64da6ce3fa2abf"
server_key_version: "encryptionkey@0"
I20260811 19:17:35.508661 8488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786475454804547-3318-0/minicluster-data/master-0/wal/instance:
uuid: "89f8f76db6c845489ec58e7f6256fe8d"
format_stamp: "Formatted at 2026-08-11 19:17:35 on dist-test-slave-t7g5"
server_key: "d3b0570cf59a808381c80bbd615e4954"
server_key_iv: "503b76bb3f5b2e1cbb64da6ce3fa2abf"
server_key_version: "encryptionkey@0"
I20260811 19:17:35.515192 8488 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260811 19:17:35.521121 8523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:17:35.523449 8488 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260811 19:17:35.523716 8488 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786475454804547-3318-0/minicluster-data/master-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786475454804547-3318-0/minicluster-data/master-0/wal
uuid: "89f8f76db6c845489ec58e7f6256fe8d"
format_stamp: "Formatted at 2026-08-11 19:17:35 on dist-test-slave-t7g5"
server_key: "d3b0570cf59a808381c80bbd615e4954"
server_key_iv: "503b76bb3f5b2e1cbb64da6ce3fa2abf"
server_key_version: "encryptionkey@0"
I20260811 19:17:35.523969 8488 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786475454804547-3318-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786475454804547-3318-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786475454804547-3318-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:17:35.532680 8488 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:17:35.538653 8488 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:17:35.539014 8488 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:17:35.562491 8488 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.190:34583
I20260811 19:17:35.562557 8634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.190:34583 every 8 connection(s)
I20260811 19:17:35.564131 8488 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
I20260811 19:17:35.568359 8635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:17:35.574179 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 8488
I20260811 19:17:35.574407 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786475454804547-3318-0/minicluster-data/master-0/wal/instance
I20260811 19:17:35.575068 3318 external_mini_cluster.cc:1428] Setting key f99a7d26dfb0aaa9abe221974b74637e
I20260811 19:17:35.585028 8635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89f8f76db6c845489ec58e7f6256fe8d: Bootstrap starting.
I20260811 19:17:35.587612 8641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:35.577610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35319 (local address 127.3.61.190:34583)
0811 19:17:35.578653 (+ 1043us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:35.578671 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:35.578740 (+ 69us) server_negotiation.cc:407] Connection header received
0811 19:17:35.582036 (+ 3296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:35.582098 (+ 62us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:35.582851 (+ 753us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:35.583519 (+ 668us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:35.584915 (+ 1396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:35.584925 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:35.584945 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:17:35.584966 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:35.585060 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:17:35.585244 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:35.585713 (+ 469us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:35.585902 (+ 189us) server_negotiation.cc:299] Negotiation successful
0811 19:17:35.586176 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":750,"thread_start_us":423,"threads_started":1}
I20260811 19:17:35.589908 8635 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 89f8f76db6c845489ec58e7f6256fe8d: Neither blocks nor log segments found. Creating new log.
I20260811 19:17:35.591281 8635 log.cc:824] T 00000000000000000000000000000000 P 89f8f76db6c845489ec58e7f6256fe8d: Log is configured to *not* fsync() on all Append() calls
I20260811 19:17:35.593362 3318 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:17:35.595451 8635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89f8f76db6c845489ec58e7f6256fe8d: No bootstrap required, opened a new log
I20260811 19:17:35.601563 8641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:35.598172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59969 (local address 127.3.61.190:34583)
0811 19:17:35.598443 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:35.598449 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:35.598645 (+ 196us) server_negotiation.cc:407] Connection header received
0811 19:17:35.598871 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:35.598879 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:35.598971 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:35.599138 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:35.600400 (+ 1262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:35.600406 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:35.600412 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:17:35.600435 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:35.600502 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:17:35.600528 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:35.600694 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:35.601125 (+ 431us) server_negotiation.cc:299] Negotiation successful
0811 19:17:35.601229 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260811 19:17:35.606305 8635 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89f8f76db6c845489ec58e7f6256fe8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89f8f76db6c845489ec58e7f6256fe8d" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 34583 } }
I20260811 19:17:35.606758 8635 raft_consensus.cc:374] T 00000000000000000000000000000000 P 89f8f76db6c845489ec58e7f6256fe8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:17:35.606860 8635 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89f8f76db6c845489ec58e7f6256fe8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89f8f76db6c845489ec58e7f6256fe8d, State: Initialized, Role: FOLLOWER
I20260811 19:17:35.607501 8635 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89f8f76db6c845489ec58e7f6256fe8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89f8f76db6c845489ec58e7f6256fe8d" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 34583 } }
I20260811 19:17:35.607748 8635 raft_consensus.cc:388] T 00000000000000000000000000000000 P 89f8f76db6c845489ec58e7f6256fe8d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:17:35.607888 8635 raft_consensus.cc:482] T 00000000000000000000000000000000 P 89f8f76db6c845489ec58e7f6256fe8d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:17:35.608053 8635 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89f8f76db6c845489ec58e7f6256fe8d [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:17:35.610551 8635 raft_consensus.cc:504] T 00000000000000000000000000000000 P 89f8f76db6c845489ec58e7f6256fe8d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89f8f76db6c845489ec58e7f6256fe8d" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 34583 } }
I20260811 19:17:35.611162 8635 leader_election.cc:302] T 00000000000000000000000000000000 P 89f8f76db6c845489ec58e7f6256fe8d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 89f8f76db6c845489ec58e7f6256fe8d; no voters:
I20260811 19:17:35.611976 8635 leader_election.cc:288] T 00000000000000000000000000000000 P 89f8f76db6c845489ec58e7f6256fe8d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:17:35.612306 8659 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 89f8f76db6c845489ec58e7f6256fe8d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:17:35.613447 8659 raft_consensus.cc:686] T 00000000000000000000000000000000 P 89f8f76db6c845489ec58e7f6256fe8d [term 1 LEADER]: Becoming Leader. State: Replica: 89f8f76db6c845489ec58e7f6256fe8d, State: Running, Role: LEADER
I20260811 19:17:35.614178 8659 consensus_queue.cc:237] T 00000000000000000000000000000000 P 89f8f76db6c845489ec58e7f6256fe8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89f8f76db6c845489ec58e7f6256fe8d" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 34583 } }
I20260811 19:17:35.614661 8635 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89f8f76db6c845489ec58e7f6256fe8d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:17:35.617616 8660 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89f8f76db6c845489ec58e7f6256fe8d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "89f8f76db6c845489ec58e7f6256fe8d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89f8f76db6c845489ec58e7f6256fe8d" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 34583 } } }
I20260811 19:17:35.617498 8661 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89f8f76db6c845489ec58e7f6256fe8d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 89f8f76db6c845489ec58e7f6256fe8d. Latest consensus state: current_term: 1 leader_uuid: "89f8f76db6c845489ec58e7f6256fe8d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89f8f76db6c845489ec58e7f6256fe8d" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 34583 } } }
I20260811 19:17:35.618077 8660 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89f8f76db6c845489ec58e7f6256fe8d [sys.catalog]: This master's current role is: LEADER
I20260811 19:17:35.618165 8661 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89f8f76db6c845489ec58e7f6256fe8d [sys.catalog]: This master's current role is: LEADER
I20260811 19:17:35.619757 8675 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:17:35.627065 8675 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:17:35.639127 8675 catalog_manager.cc:1347] Generated new cluster ID: 7175c3f6e7bd465cb2e3f56279ceeffa
I20260811 19:17:35.639298 8675 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:17:35.660660 8675 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:17:35.661242 8675 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:17:35.672294 8675 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 89f8f76db6c845489ec58e7f6256fe8d: Generated new TSK 0
I20260811 19:17:35.673604 8675 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:17:35.688783 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 8488
W20260811 19:17:35.690543 8625 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:35319
W20260811 19:17:35.691324 3318 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 8488: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 19:17:35.691480 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 8488
2026-08-11T19:17:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2485 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786475454804547-3318-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:17:35 dist-test-slave-t7g5 krb5kdc[8694](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:17:35 dist-test-slave-t7g5 krb5kdc[8694](info): set up 2 sockets
Aug 11 19:17:35 dist-test-slave-t7g5 krb5kdc[8694](info): commencing operation
krb5kdc: starting...
W20260811 19:17:37.048074 3318 mini_kdc.cc:121] Time spent starting KDC: real 1.303s user 0.000s sys 0.072s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Aug 11 19:17:37 dist-test-slave-t7g5 krb5kdc[8694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475857, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:17:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:17:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.61.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab.
Entry for principal kudu/127.3.61.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab.
I20260811 19:17:37.468140 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786475454804547-3318-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786475454804547-3318-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786475454804547-3318-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:37773
--webserver_interface=127.3.61.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:46037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.61.190:37773
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:17:37.900943 8713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:17:37.901321 8713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:17:37.901448 8713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:17:37.909715 8713 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:17:37.909901 8713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:17:37.909957 8713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:17:37.910056 8713 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:17:37.910122 8713 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:17:37.921427 8713 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:46037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786475454804547-3318-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786475454804547-3318-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.61.190:37773
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:37773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.61.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786475454804547-3318-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:17:37.923094 8713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:17:37.924685 8713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:17:37.939378 8722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:17:37.939962 8728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:17:37.940089 8723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:17:37.941895 8713 server_base.cc:1034] running on GCE node
I20260811 19:17:37.943300 8713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:17:37.945531 8713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:17:37.946851 8713 hybrid_clock.cc:648] HybridClock initialized: now 1786475857946775 us; error 75 us; skew 500 ppm
I20260811 19:17:37.947317 8713 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:17:37 dist-test-slave-t7g5 krb5kdc[8694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475857, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:17:37.951650 8713 init.cc:377] Logged in from keytab as kudu/127.3.61.190@KRBTEST.COM (short username kudu)
I20260811 19:17:37.953809 8713 webserver.cc:460] Webserver started at http://127.3.61.190:44497/ using document root <none> and password file <none>
I20260811 19:17:37.954674 8713 fs_manager.cc:359] Metadata directory not provided
I20260811 19:17:37.954808 8713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:17:37.955111 8713 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:17:37.958791 8713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786475454804547-3318-0/minicluster-data/master-0/data/instance:
uuid: "e00c07c4d5b146d7a1d6b4383c5f65ab"
format_stamp: "Formatted at 2026-08-11 19:17:37 on dist-test-slave-t7g5"
server_key: "5d0f00eec9202d6c986882316c396c82"
server_key_iv: "a368d7364e6b8ec3805da32152afe844"
server_key_version: "encryptionkey@0"
I20260811 19:17:37.959688 8713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786475454804547-3318-0/minicluster-data/master-0/wal/instance:
uuid: "e00c07c4d5b146d7a1d6b4383c5f65ab"
format_stamp: "Formatted at 2026-08-11 19:17:37 on dist-test-slave-t7g5"
server_key: "5d0f00eec9202d6c986882316c396c82"
server_key_iv: "a368d7364e6b8ec3805da32152afe844"
server_key_version: "encryptionkey@0"
I20260811 19:17:37.966524 8713 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260811 19:17:37.972291 8746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:17:37.974459 8713 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.003s
I20260811 19:17:37.974723 8713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786475454804547-3318-0/minicluster-data/master-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786475454804547-3318-0/minicluster-data/master-0/wal
uuid: "e00c07c4d5b146d7a1d6b4383c5f65ab"
format_stamp: "Formatted at 2026-08-11 19:17:37 on dist-test-slave-t7g5"
server_key: "5d0f00eec9202d6c986882316c396c82"
server_key_iv: "a368d7364e6b8ec3805da32152afe844"
server_key_version: "encryptionkey@0"
I20260811 19:17:37.974967 8713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786475454804547-3318-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786475454804547-3318-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786475454804547-3318-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:17:37.991040 8713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:17:37.996222 8713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:17:37.996590 8713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:17:38.021715 8713 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.190:37773
I20260811 19:17:38.021807 8839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.190:37773 every 8 connection(s)
I20260811 19:17:38.023372 8713 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
I20260811 19:17:38.027684 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 8713
I20260811 19:17:38.027793 8843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:17:38.027912 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786475454804547-3318-0/minicluster-data/master-0/wal/instance
I20260811 19:17:38.028477 3318 external_mini_cluster.cc:1428] Setting key 77252ac4e30a0746b242a81b461346a8
I20260811 19:17:38.045840 8843 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e00c07c4d5b146d7a1d6b4383c5f65ab: Bootstrap starting.
Aug 11 19:17:38 dist-test-slave-t7g5 krb5kdc[8694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475857, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:17:38.050072 8843 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e00c07c4d5b146d7a1d6b4383c5f65ab: Neither blocks nor log segments found. Creating new log.
I20260811 19:17:38.051517 8843 log.cc:824] T 00000000000000000000000000000000 P e00c07c4d5b146d7a1d6b4383c5f65ab: Log is configured to *not* fsync() on all Append() calls
I20260811 19:17:38.056041 8843 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e00c07c4d5b146d7a1d6b4383c5f65ab: No bootstrap required, opened a new log
I20260811 19:17:38.058665 8848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:38.030927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42871 (local address 127.3.61.190:37773)
0811 19:17:38.032098 (+ 1171us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:38.032117 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:38.032192 (+ 75us) server_negotiation.cc:407] Connection header received
0811 19:17:38.034920 (+ 2728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:38.034970 (+ 50us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:38.035625 (+ 655us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:38.036064 (+ 439us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:38.038263 (+ 2199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:38.039758 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:38.041826 (+ 2068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:38.042729 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:38.042986 (+ 257us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:38.049020 (+ 6034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:38.049030 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:38.049054 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:17:38.049112 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:38.052995 (+ 3883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:38.054493 (+ 1498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:38.054503 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:38.054511 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:38.054760 (+ 249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:38.055392 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:38.055397 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:38.055400 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:38.056045 (+ 645us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:38.056351 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:38.056974 (+ 623us) server_negotiation.cc:299] Negotiation successful
0811 19:17:38.057443 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":829,"thread_start_us":475,"threads_started":1}
I20260811 19:17:38.064146 3318 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:17:38.066706 8843 raft_consensus.cc:348] T 00000000000000000000000000000000 P e00c07c4d5b146d7a1d6b4383c5f65ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e00c07c4d5b146d7a1d6b4383c5f65ab" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 37773 } }
I20260811 19:17:38.067636 8843 raft_consensus.cc:374] T 00000000000000000000000000000000 P e00c07c4d5b146d7a1d6b4383c5f65ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:17:38.067782 8843 raft_consensus.cc:729] T 00000000000000000000000000000000 P e00c07c4d5b146d7a1d6b4383c5f65ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e00c07c4d5b146d7a1d6b4383c5f65ab, State: Initialized, Role: FOLLOWER
I20260811 19:17:38.068886 8843 consensus_queue.cc:260] T 00000000000000000000000000000000 P e00c07c4d5b146d7a1d6b4383c5f65ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e00c07c4d5b146d7a1d6b4383c5f65ab" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 37773 } }
I20260811 19:17:38.069314 8843 raft_consensus.cc:388] T 00000000000000000000000000000000 P e00c07c4d5b146d7a1d6b4383c5f65ab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:17:38.069500 8843 raft_consensus.cc:482] T 00000000000000000000000000000000 P e00c07c4d5b146d7a1d6b4383c5f65ab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:17:38.069774 8843 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e00c07c4d5b146d7a1d6b4383c5f65ab [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:17:38.073884 8843 raft_consensus.cc:504] T 00000000000000000000000000000000 P e00c07c4d5b146d7a1d6b4383c5f65ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e00c07c4d5b146d7a1d6b4383c5f65ab" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 37773 } }
I20260811 19:17:38.075258 8843 leader_election.cc:302] T 00000000000000000000000000000000 P e00c07c4d5b146d7a1d6b4383c5f65ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e00c07c4d5b146d7a1d6b4383c5f65ab; no voters:
I20260811 19:17:38.076166 8843 leader_election.cc:288] T 00000000000000000000000000000000 P e00c07c4d5b146d7a1d6b4383c5f65ab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:17:38.076324 8859 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e00c07c4d5b146d7a1d6b4383c5f65ab [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:17:38.078197 8859 raft_consensus.cc:686] T 00000000000000000000000000000000 P e00c07c4d5b146d7a1d6b4383c5f65ab [term 1 LEADER]: Becoming Leader. State: Replica: e00c07c4d5b146d7a1d6b4383c5f65ab, State: Running, Role: LEADER
I20260811 19:17:38.079125 8859 consensus_queue.cc:237] T 00000000000000000000000000000000 P e00c07c4d5b146d7a1d6b4383c5f65ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e00c07c4d5b146d7a1d6b4383c5f65ab" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 37773 } }
I20260811 19:17:38.079763 8843 sys_catalog.cc:564] T 00000000000000000000000000000000 P e00c07c4d5b146d7a1d6b4383c5f65ab [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:17:38.083884 8860 sys_catalog.cc:455] T 00000000000000000000000000000000 P e00c07c4d5b146d7a1d6b4383c5f65ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e00c07c4d5b146d7a1d6b4383c5f65ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e00c07c4d5b146d7a1d6b4383c5f65ab" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 37773 } } }
I20260811 19:17:38.085055 8860 sys_catalog.cc:458] T 00000000000000000000000000000000 P e00c07c4d5b146d7a1d6b4383c5f65ab [sys.catalog]: This master's current role is: LEADER
I20260811 19:17:38.083696 8863 sys_catalog.cc:455] T 00000000000000000000000000000000 P e00c07c4d5b146d7a1d6b4383c5f65ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader e00c07c4d5b146d7a1d6b4383c5f65ab. Latest consensus state: current_term: 1 leader_uuid: "e00c07c4d5b146d7a1d6b4383c5f65ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e00c07c4d5b146d7a1d6b4383c5f65ab" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 37773 } } }
I20260811 19:17:38.085525 8863 sys_catalog.cc:458] T 00000000000000000000000000000000 P e00c07c4d5b146d7a1d6b4383c5f65ab [sys.catalog]: This master's current role is: LEADER
I20260811 19:17:38.091648 8875 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:17:38.092269 8848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:38.071100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48449 (local address 127.3.61.190:37773)
0811 19:17:38.071393 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:38.071401 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:38.071929 (+ 528us) server_negotiation.cc:407] Connection header received
0811 19:17:38.072074 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:38.072079 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:38.072181 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:38.072315 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:38.075003 (+ 2688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:38.075880 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:38.077839 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:38.078612 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:38.078764 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:38.082235 (+ 3471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:38.082244 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:38.082250 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:17:38.082305 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:38.087266 (+ 4961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:38.088560 (+ 1294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:38.088567 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:38.088572 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:38.088764 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:38.089326 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:38.089332 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:38.089336 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:38.089874 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:38.091466 (+ 1592us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:38.091641 (+ 175us) server_negotiation.cc:299] Negotiation successful
0811 19:17:38.091927 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260811 19:17:38.102037 8875 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:17:38.115619 8875 catalog_manager.cc:1347] Generated new cluster ID: 032fb5f0c0204f069f21cc3c33dd732a
I20260811 19:17:38.115789 8875 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:17:38.138314 8875 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:17:38.140568 8875 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:17:38.151671 8875 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e00c07c4d5b146d7a1d6b4383c5f65ab: Generated new TSK 0
I20260811 19:17:38.152721 8875 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:17:38.167515 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 8713
I20260811 19:17:38.169027 8832 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:17:38.270001 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 8713
2026-08-11T19:17:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2577 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786475454804547-3318-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:17:38 dist-test-slave-t7g5 krb5kdc[8898](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:17:38 dist-test-slave-t7g5 krb5kdc[8898](info): set up 2 sockets
Aug 11 19:17:38 dist-test-slave-t7g5 krb5kdc[8898](info): commencing operation
krb5kdc: starting...
W20260811 19:17:39.671089 3318 mini_kdc.cc:121] Time spent starting KDC: real 1.347s user 0.000s sys 0.079s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Aug 11 19:17:39 dist-test-slave-t7g5 krb5kdc[8898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475859, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:17:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:17:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.61.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab.
Entry for principal kudu/127.3.61.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab.
I20260811 19:17:40.115271 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786475454804547-3318-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786475454804547-3318-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786475454804547-3318-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:36949
--webserver_interface=127.3.61.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:44721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.61.190:36949
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:17:40.582602 8924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:17:40.582990 8924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:17:40.583125 8924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:17:40.593047 8924 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:17:40.593264 8924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:17:40.593346 8924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:17:40.593451 8924 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:17:40.593513 8924 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:17:40.606842 8924 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:44721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786475454804547-3318-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786475454804547-3318-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.61.190:36949
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:36949
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.61.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786475454804547-3318-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:17:40.608479 8924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:17:40.610268 8924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:17:40.626443 8936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:17:40.626375 8943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:17:40.626420 8937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:17:40.628399 8924 server_base.cc:1034] running on GCE node
I20260811 19:17:40.629846 8924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:17:40.631750 8924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:17:40.633064 8924 hybrid_clock.cc:648] HybridClock initialized: now 1786475860632986 us; error 77 us; skew 500 ppm
I20260811 19:17:40.633539 8924 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:17:40 dist-test-slave-t7g5 krb5kdc[8898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475860, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:17:40.642302 8924 init.cc:377] Logged in from keytab as kudu/127.3.61.190@KRBTEST.COM (short username kudu)
I20260811 19:17:40.644527 8924 webserver.cc:460] Webserver started at http://127.3.61.190:40975/ using document root <none> and password file <none>
I20260811 19:17:40.645440 8924 fs_manager.cc:359] Metadata directory not provided
I20260811 19:17:40.645602 8924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:17:40.646019 8924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:17:40.649737 8924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786475454804547-3318-0/minicluster-data/master-0/data/instance:
uuid: "dd9e80e55e794123b7a64b3a47fb3a58"
format_stamp: "Formatted at 2026-08-11 19:17:40 on dist-test-slave-t7g5"
server_key: "951a8113dda68453ee12313671ac7861"
server_key_iv: "817829ee8d7e175c12847e82e2594244"
server_key_version: "encryptionkey@0"
I20260811 19:17:40.650810 8924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786475454804547-3318-0/minicluster-data/master-0/wal/instance:
uuid: "dd9e80e55e794123b7a64b3a47fb3a58"
format_stamp: "Formatted at 2026-08-11 19:17:40 on dist-test-slave-t7g5"
server_key: "951a8113dda68453ee12313671ac7861"
server_key_iv: "817829ee8d7e175c12847e82e2594244"
server_key_version: "encryptionkey@0"
I20260811 19:17:40.659240 8924 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.007s sys 0.001s
I20260811 19:17:40.665238 8956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:17:40.667789 8924 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260811 19:17:40.668098 8924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786475454804547-3318-0/minicluster-data/master-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786475454804547-3318-0/minicluster-data/master-0/wal
uuid: "dd9e80e55e794123b7a64b3a47fb3a58"
format_stamp: "Formatted at 2026-08-11 19:17:40 on dist-test-slave-t7g5"
server_key: "951a8113dda68453ee12313671ac7861"
server_key_iv: "817829ee8d7e175c12847e82e2594244"
server_key_version: "encryptionkey@0"
I20260811 19:17:40.668412 8924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786475454804547-3318-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786475454804547-3318-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786475454804547-3318-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:17:40.703094 8924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:17:40.708174 8924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:17:40.708498 8924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:17:40.734109 8924 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.190:36949
I20260811 19:17:40.734164 9061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.190:36949 every 8 connection(s)
I20260811 19:17:40.736274 8924 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
I20260811 19:17:40.739171 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 8924
I20260811 19:17:40.739377 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786475454804547-3318-0/minicluster-data/master-0/wal/instance
I20260811 19:17:40.739893 3318 external_mini_cluster.cc:1428] Setting key bf30ab39f78cae79c4381b1c5b86524b
I20260811 19:17:40.741403 9063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:17:40.758342 9063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd9e80e55e794123b7a64b3a47fb3a58: Bootstrap starting.
I20260811 19:17:40.758864 9073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:40.742109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49909 (local address 127.3.61.190:36949)
0811 19:17:40.743308 (+ 1199us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:40.743326 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:40.743606 (+ 280us) server_negotiation.cc:407] Connection header received
0811 19:17:40.746087 (+ 2481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:40.746125 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:40.746826 (+ 701us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:40.747365 (+ 539us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:40.748307 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:40.750269 (+ 1962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:40.752438 (+ 2169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:40.753314 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:40.753515 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:40.755387 (+ 1872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:40.755403 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:40.755434 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:17:40.756362 (+ 928us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:40.756496 (+ 134us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:17:40.756534 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:40.756697 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:40.757264 (+ 567us) server_negotiation.cc:299] Negotiation successful
0811 19:17:40.757611 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":876,"thread_start_us":453,"threads_started":1}
I20260811 19:17:40.762446 9063 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd9e80e55e794123b7a64b3a47fb3a58: Neither blocks nor log segments found. Creating new log.
I20260811 19:17:40.763795 9063 log.cc:824] T 00000000000000000000000000000000 P dd9e80e55e794123b7a64b3a47fb3a58: Log is configured to *not* fsync() on all Append() calls
I20260811 19:17:40.764195 3318 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:17:40.767552 9063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd9e80e55e794123b7a64b3a47fb3a58: No bootstrap required, opened a new log
I20260811 19:17:40.775985 9073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:40.769318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50725 (local address 127.3.61.190:36949)
0811 19:17:40.769538 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:40.769545 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:40.770022 (+ 477us) server_negotiation.cc:407] Connection header received
0811 19:17:40.770232 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:40.770236 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:40.770321 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:40.770420 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:40.771031 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:40.771821 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:40.773235 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:40.773798 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:40.773921 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:40.775081 (+ 1160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:40.775086 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:40.775090 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:17:40.775115 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:40.775168 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:17:40.775193 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:40.775287 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:40.775534 (+ 247us) server_negotiation.cc:299] Negotiation successful
0811 19:17:40.775646 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 19:17:40.776580 9063 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd9e80e55e794123b7a64b3a47fb3a58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd9e80e55e794123b7a64b3a47fb3a58" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 36949 } }
I20260811 19:17:40.777021 9063 raft_consensus.cc:374] T 00000000000000000000000000000000 P dd9e80e55e794123b7a64b3a47fb3a58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:17:40.777125 9063 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd9e80e55e794123b7a64b3a47fb3a58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd9e80e55e794123b7a64b3a47fb3a58, State: Initialized, Role: FOLLOWER
I20260811 19:17:40.777872 9063 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd9e80e55e794123b7a64b3a47fb3a58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd9e80e55e794123b7a64b3a47fb3a58" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 36949 } }
I20260811 19:17:40.778136 9063 raft_consensus.cc:388] T 00000000000000000000000000000000 P dd9e80e55e794123b7a64b3a47fb3a58 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:17:40.778281 9063 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd9e80e55e794123b7a64b3a47fb3a58 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:17:40.778473 9063 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd9e80e55e794123b7a64b3a47fb3a58 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:17:40.782014 9063 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd9e80e55e794123b7a64b3a47fb3a58 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd9e80e55e794123b7a64b3a47fb3a58" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 36949 } }
I20260811 19:17:40.782672 9063 leader_election.cc:302] T 00000000000000000000000000000000 P dd9e80e55e794123b7a64b3a47fb3a58 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dd9e80e55e794123b7a64b3a47fb3a58; no voters:
I20260811 19:17:40.783480 9063 leader_election.cc:288] T 00000000000000000000000000000000 P dd9e80e55e794123b7a64b3a47fb3a58 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:17:40.783769 9083 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dd9e80e55e794123b7a64b3a47fb3a58 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:17:40.784691 9083 raft_consensus.cc:686] T 00000000000000000000000000000000 P dd9e80e55e794123b7a64b3a47fb3a58 [term 1 LEADER]: Becoming Leader. State: Replica: dd9e80e55e794123b7a64b3a47fb3a58, State: Running, Role: LEADER
I20260811 19:17:40.785429 9083 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd9e80e55e794123b7a64b3a47fb3a58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd9e80e55e794123b7a64b3a47fb3a58" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 36949 } }
I20260811 19:17:40.786118 9063 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd9e80e55e794123b7a64b3a47fb3a58 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:17:40.789166 9088 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd9e80e55e794123b7a64b3a47fb3a58 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd9e80e55e794123b7a64b3a47fb3a58. Latest consensus state: current_term: 1 leader_uuid: "dd9e80e55e794123b7a64b3a47fb3a58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd9e80e55e794123b7a64b3a47fb3a58" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 36949 } } }
I20260811 19:17:40.789726 9088 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd9e80e55e794123b7a64b3a47fb3a58 [sys.catalog]: This master's current role is: LEADER
I20260811 19:17:40.790194 9086 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd9e80e55e794123b7a64b3a47fb3a58 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dd9e80e55e794123b7a64b3a47fb3a58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd9e80e55e794123b7a64b3a47fb3a58" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 36949 } } }
I20260811 19:17:40.790416 9086 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd9e80e55e794123b7a64b3a47fb3a58 [sys.catalog]: This master's current role is: LEADER
I20260811 19:17:40.792500 9094 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:17:40.800002 9094 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:17:40.812731 9094 catalog_manager.cc:1347] Generated new cluster ID: 40b856a16e3b4874ae92b39f4e61cb29
I20260811 19:17:40.812923 9094 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:17:40.837201 9094 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:17:40.838853 9094 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:17:40.847462 9094 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dd9e80e55e794123b7a64b3a47fb3a58: Generated new TSK 0
I20260811 19:17:40.848440 9094 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:17:40.865288 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 8924
W20260811 19:17:40.866671 9054 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:49909
W20260811 19:17:40.867559 3318 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 8924: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 19:17:40.867703 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 8924
2026-08-11T19:17:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2603 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786475454804547-3318-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:17:41 dist-test-slave-t7g5 krb5kdc[9120](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:17:41 dist-test-slave-t7g5 krb5kdc[9120](info): set up 2 sockets
Aug 11 19:17:41 dist-test-slave-t7g5 krb5kdc[9120](info): commencing operation
krb5kdc: starting...
W20260811 19:17:42.260730 3318 mini_kdc.cc:121] Time spent starting KDC: real 1.334s user 0.003s sys 0.071s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Aug 11 19:17:42 dist-test-slave-t7g5 krb5kdc[9120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:17:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:17:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.61.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab.
Entry for principal kudu/127.3.61.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab.
I20260811 19:17:42.719079 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786475454804547-3318-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786475454804547-3318-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786475454804547-3318-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:35027
--webserver_interface=127.3.61.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:33943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.61.190:35027
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:17:43.138151 9139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:17:43.138514 9139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:17:43.138615 9139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:17:43.147130 9139 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:17:43.147310 9139 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:17:43.147363 9139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:17:43.147449 9139 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:17:43.147498 9139 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:17:43.158974 9139 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:33943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786475454804547-3318-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786475454804547-3318-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.61.190:35027
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:35027
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.61.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786475454804547-3318-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:17:43.160454 9139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:17:43.161993 9139 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:17:43.177004 9153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:17:43.176977 9152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:17:43.177228 9156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:17:43.178004 9139 server_base.cc:1034] running on GCE node
I20260811 19:17:43.178970 9139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:17:43.180683 9139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:17:43.181919 9139 hybrid_clock.cc:648] HybridClock initialized: now 1786475863181865 us; error 48 us; skew 500 ppm
I20260811 19:17:43.182550 9139 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:17:43 dist-test-slave-t7g5 krb5kdc[9120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475863, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:17:43.187561 9139 init.cc:377] Logged in from keytab as kudu/127.3.61.190@KRBTEST.COM (short username kudu)
I20260811 19:17:43.190052 9139 webserver.cc:460] Webserver started at http://127.3.61.190:43537/ using document root <none> and password file <none>
I20260811 19:17:43.191114 9139 fs_manager.cc:359] Metadata directory not provided
I20260811 19:17:43.191280 9139 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:17:43.191694 9139 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:17:43.195911 9139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786475454804547-3318-0/minicluster-data/master-0/data/instance:
uuid: "d3f4ec7aefc44ed3b40c9f7bd33c6209"
format_stamp: "Formatted at 2026-08-11 19:17:43 on dist-test-slave-t7g5"
server_key: "dd1da01954ac465a0ff72ee8796d4062"
server_key_iv: "8ddfebdfb4d13f0dc174f839c9ae392b"
server_key_version: "encryptionkey@0"
I20260811 19:17:43.196933 9139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786475454804547-3318-0/minicluster-data/master-0/wal/instance:
uuid: "d3f4ec7aefc44ed3b40c9f7bd33c6209"
format_stamp: "Formatted at 2026-08-11 19:17:43 on dist-test-slave-t7g5"
server_key: "dd1da01954ac465a0ff72ee8796d4062"
server_key_iv: "8ddfebdfb4d13f0dc174f839c9ae392b"
server_key_version: "encryptionkey@0"
I20260811 19:17:43.203363 9139 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.008s
I20260811 19:17:43.208158 9177 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:17:43.210582 9139 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260811 19:17:43.210940 9139 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786475454804547-3318-0/minicluster-data/master-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786475454804547-3318-0/minicluster-data/master-0/wal
uuid: "d3f4ec7aefc44ed3b40c9f7bd33c6209"
format_stamp: "Formatted at 2026-08-11 19:17:43 on dist-test-slave-t7g5"
server_key: "dd1da01954ac465a0ff72ee8796d4062"
server_key_iv: "8ddfebdfb4d13f0dc174f839c9ae392b"
server_key_version: "encryptionkey@0"
I20260811 19:17:43.211261 9139 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786475454804547-3318-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786475454804547-3318-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786475454804547-3318-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:17:43.220474 9139 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:17:43.226253 9139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:17:43.226930 9139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:17:43.252041 9139 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.190:35027
I20260811 19:17:43.252071 9291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.190:35027 every 8 connection(s)
I20260811 19:17:43.254225 9139 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
I20260811 19:17:43.257850 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 9139
I20260811 19:17:43.258077 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786475454804547-3318-0/minicluster-data/master-0/wal/instance
I20260811 19:17:43.258553 3318 external_mini_cluster.cc:1428] Setting key f7378a337e866c7025dd04c253476a48
I20260811 19:17:43.259572 9292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:17:43.271600 9301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:43.261140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35577 (local address 127.3.61.190:35027)
0811 19:17:43.262116 (+ 976us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:43.262133 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:43.262193 (+ 60us) server_negotiation.cc:407] Connection header received
0811 19:17:43.265699 (+ 3506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:43.265763 (+ 64us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:43.266641 (+ 878us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:43.267269 (+ 628us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:43.268679 (+ 1410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:43.268691 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:43.268727 (+ 36us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:17:43.268761 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:43.268876 (+ 115us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:17:43.268928 (+ 52us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:43.269298 (+ 370us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:43.269503 (+ 205us) server_negotiation.cc:299] Negotiation successful
0811 19:17:43.269862 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":684,"thread_start_us":434,"threads_started":1}
I20260811 19:17:43.277196 3318 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:17:43.279618 9292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3f4ec7aefc44ed3b40c9f7bd33c6209: Bootstrap starting.
I20260811 19:17:43.285552 9292 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d3f4ec7aefc44ed3b40c9f7bd33c6209: Neither blocks nor log segments found. Creating new log.
I20260811 19:17:43.287271 9301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:43.284183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44629 (local address 127.3.61.190:35027)
0811 19:17:43.284544 (+ 361us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:43.284552 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:43.284588 (+ 36us) server_negotiation.cc:407] Connection header received
0811 19:17:43.284798 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:43.284804 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:43.284896 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:43.285065 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:43.286381 (+ 1316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:43.286386 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:43.286391 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 19:17:43.286411 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:43.286471 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 19:17:43.286494 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:43.286635 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:43.286835 (+ 200us) server_negotiation.cc:299] Negotiation successful
0811 19:17:43.286913 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260811 19:17:43.287405 9292 log.cc:824] T 00000000000000000000000000000000 P d3f4ec7aefc44ed3b40c9f7bd33c6209: Log is configured to *not* fsync() on all Append() calls
I20260811 19:17:43.291371 9292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3f4ec7aefc44ed3b40c9f7bd33c6209: No bootstrap required, opened a new log
I20260811 19:17:43.300634 9292 raft_consensus.cc:348] T 00000000000000000000000000000000 P d3f4ec7aefc44ed3b40c9f7bd33c6209 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3f4ec7aefc44ed3b40c9f7bd33c6209" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35027 } }
I20260811 19:17:43.301147 9292 raft_consensus.cc:374] T 00000000000000000000000000000000 P d3f4ec7aefc44ed3b40c9f7bd33c6209 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 19:17:43.301291 9292 raft_consensus.cc:729] T 00000000000000000000000000000000 P d3f4ec7aefc44ed3b40c9f7bd33c6209 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3f4ec7aefc44ed3b40c9f7bd33c6209, State: Initialized, Role: FOLLOWER
I20260811 19:17:43.302029 9292 consensus_queue.cc:260] T 00000000000000000000000000000000 P d3f4ec7aefc44ed3b40c9f7bd33c6209 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3f4ec7aefc44ed3b40c9f7bd33c6209" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35027 } }
I20260811 19:17:43.302305 9292 raft_consensus.cc:388] T 00000000000000000000000000000000 P d3f4ec7aefc44ed3b40c9f7bd33c6209 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 19:17:43.302455 9292 raft_consensus.cc:482] T 00000000000000000000000000000000 P d3f4ec7aefc44ed3b40c9f7bd33c6209 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 19:17:43.302630 9292 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d3f4ec7aefc44ed3b40c9f7bd33c6209 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:17:43.305097 9292 raft_consensus.cc:504] T 00000000000000000000000000000000 P d3f4ec7aefc44ed3b40c9f7bd33c6209 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3f4ec7aefc44ed3b40c9f7bd33c6209" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35027 } }
I20260811 19:17:43.305775 9292 leader_election.cc:302] T 00000000000000000000000000000000 P d3f4ec7aefc44ed3b40c9f7bd33c6209 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d3f4ec7aefc44ed3b40c9f7bd33c6209; no voters:
I20260811 19:17:43.306538 9292 leader_election.cc:288] T 00000000000000000000000000000000 P d3f4ec7aefc44ed3b40c9f7bd33c6209 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 19:17:43.306814 9312 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d3f4ec7aefc44ed3b40c9f7bd33c6209 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:17:43.307749 9312 raft_consensus.cc:686] T 00000000000000000000000000000000 P d3f4ec7aefc44ed3b40c9f7bd33c6209 [term 1 LEADER]: Becoming Leader. State: Replica: d3f4ec7aefc44ed3b40c9f7bd33c6209, State: Running, Role: LEADER
I20260811 19:17:43.308460 9312 consensus_queue.cc:237] T 00000000000000000000000000000000 P d3f4ec7aefc44ed3b40c9f7bd33c6209 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3f4ec7aefc44ed3b40c9f7bd33c6209" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35027 } }
I20260811 19:17:43.308972 9292 sys_catalog.cc:564] T 00000000000000000000000000000000 P d3f4ec7aefc44ed3b40c9f7bd33c6209 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:17:43.311707 9313 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3f4ec7aefc44ed3b40c9f7bd33c6209 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d3f4ec7aefc44ed3b40c9f7bd33c6209" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3f4ec7aefc44ed3b40c9f7bd33c6209" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35027 } } }
I20260811 19:17:43.312155 9313 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3f4ec7aefc44ed3b40c9f7bd33c6209 [sys.catalog]: This master's current role is: LEADER
I20260811 19:17:43.311607 9314 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3f4ec7aefc44ed3b40c9f7bd33c6209 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d3f4ec7aefc44ed3b40c9f7bd33c6209. Latest consensus state: current_term: 1 leader_uuid: "d3f4ec7aefc44ed3b40c9f7bd33c6209" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3f4ec7aefc44ed3b40c9f7bd33c6209" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 35027 } } }
I20260811 19:17:43.312310 9314 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3f4ec7aefc44ed3b40c9f7bd33c6209 [sys.catalog]: This master's current role is: LEADER
I20260811 19:17:43.313366 9323 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:17:43.319324 9323 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:17:43.332682 9323 catalog_manager.cc:1347] Generated new cluster ID: 959be45163544f3e90c7c24285e081c6
I20260811 19:17:43.332852 9323 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:17:43.343833 9323 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:17:43.344372 9323 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:17:43.365183 9323 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d3f4ec7aefc44ed3b40c9f7bd33c6209: Generated new TSK 0
I20260811 19:17:43.366461 9323 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 19:17:43.381699 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 9139
W20260811 19:17:43.382979 9286 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:35577
W20260811 19:17:43.384083 3318 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 9139: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 19:17:43.384234 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 9139
2026-08-11T19:17:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2508 ms)
[----------] 18 tests from AuthTokenIssuingTest (44967 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:17:43 dist-test-slave-t7g5 krb5kdc[9350](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:17:43 dist-test-slave-t7g5 krb5kdc[9350](info): set up 2 sockets
Aug 11 19:17:43 dist-test-slave-t7g5 krb5kdc[9350](info): commencing operation
krb5kdc: starting...
W20260811 19:17:44.768304 3318 mini_kdc.cc:121] Time spent starting KDC: real 1.333s user 0.001s sys 0.069s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Aug 11 19:17:45 dist-test-slave-t7g5 krb5kdc[9350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:17:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:17:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.61.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab.
Entry for principal kudu/127.3.61.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab.
I20260811 19:17:45.194288 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:40729
--webserver_interface=127.3.61.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:36683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.61.190:40729,127.3.61.189:34651,127.3.61.188:41707
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:17:45.604206 9369 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 19:17:45.604589 9369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:17:45.604665 9369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:17:45.604750 9369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:17:45.613763 9369 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:17:45.613922 9369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:17:45.613989 9369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:17:45.614041 9369 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:17:45.614110 9369 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:17:45.625624 9369 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:36683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.61.190:40729,127.3.61.189:34651,127.3.61.188:41707
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:40729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.61.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:17:45.627091 9369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:17:45.628707 9369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:17:45.643800 9377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:17:45.643723 9376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:17:45.644364 9382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:17:45.644435 9369 server_base.cc:1034] running on GCE node
I20260811 19:17:45.645357 9369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:17:45.647172 9369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:17:45.648553 9369 hybrid_clock.cc:648] HybridClock initialized: now 1786475865648411 us; error 53 us; skew 500 ppm
I20260811 19:17:45.648960 9369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:17:45 dist-test-slave-t7g5 krb5kdc[9350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475865, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:17:45.658241 9369 init.cc:377] Logged in from keytab as kudu/127.3.61.190@KRBTEST.COM (short username kudu)
I20260811 19:17:45.660604 9369 webserver.cc:460] Webserver started at http://127.3.61.190:44921/ using document root <none> and password file <none>
I20260811 19:17:45.661612 9369 fs_manager.cc:359] Metadata directory not provided
I20260811 19:17:45.661790 9369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:17:45.662132 9369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:17:45.665586 9369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/minicluster-data/master-0/data/instance:
uuid: "fc2444314f864d188196194f1cc75c46"
format_stamp: "Formatted at 2026-08-11 19:17:45 on dist-test-slave-t7g5"
server_key: "3419cd32d1939ffe8636b5d33eca6724"
server_key_iv: "daae6939db871324aa7c7bc335546a7a"
server_key_version: "encryptionkey@0"
I20260811 19:17:45.666489 9369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/minicluster-data/master-0/wal/instance:
uuid: "fc2444314f864d188196194f1cc75c46"
format_stamp: "Formatted at 2026-08-11 19:17:45 on dist-test-slave-t7g5"
server_key: "3419cd32d1939ffe8636b5d33eca6724"
server_key_iv: "daae6939db871324aa7c7bc335546a7a"
server_key_version: "encryptionkey@0"
I20260811 19:17:45.674304 9369 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.005s
I20260811 19:17:45.678920 9401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:17:45.680545 9369 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260811 19:17:45.680768 9369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/minicluster-data/master-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/minicluster-data/master-0/wal
uuid: "fc2444314f864d188196194f1cc75c46"
format_stamp: "Formatted at 2026-08-11 19:17:45 on dist-test-slave-t7g5"
server_key: "3419cd32d1939ffe8636b5d33eca6724"
server_key_iv: "daae6939db871324aa7c7bc335546a7a"
server_key_version: "encryptionkey@0"
I20260811 19:17:45.681041 9369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:17:45.705811 9369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:17:45.710350 9369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:17:45.710640 9369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:17:45.733853 9369 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.190:40729
I20260811 19:17:45.733840 9518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.190:40729 every 8 connection(s)
I20260811 19:17:45.735443 9369 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
I20260811 19:17:45.737121 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 9369
I20260811 19:17:45.737329 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/minicluster-data/master-0/wal/instance
I20260811 19:17:45.738037 3318 external_mini_cluster.cc:1428] Setting key 1e33e718fbb9b5d4ac1c9ff914e04d0e
I20260811 19:17:45.739748 9520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:17:45.748457 9520 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.61.190" port: 40729 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 19:17:45 dist-test-slave-t7g5 krb5kdc[9350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475865, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.190@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:17:45.778352 9522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:45.753529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:40729 (local address 127.0.0.1:54030)
0811 19:17:45.754692 (+ 1163us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:45.754756 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 19:17:45.757377 (+ 2621us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:17:45.758833 (+ 1456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:17:45.758839 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:17:45.759500 (+ 661us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:17:45.759881 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:17:45.759902 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:17:45.762300 (+ 2398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:17:45.762305 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:17:45.763493 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:17:45.763508 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:17:45.764833 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:17:45.764843 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:17:45.765136 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:45.766355 (+ 1219us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:17:45.766385 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:17:45.769295 (+ 2910us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:17:45.773893 (+ 4598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:17:45.773905 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:17:45.773917 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:17:45.774376 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:17:45.774913 (+ 537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:17:45.774917 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:17:45.774920 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:17:45.775109 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:17:45.776224 (+ 1115us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:17:45.776231 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:17:45.776464 (+ 233us) client_negotiation.cc:769] Sending connection context
0811 19:17:45.776700 (+ 236us) client_negotiation.cc:240] Negotiation successful
0811 19:17:45.777167 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":792,"thread_start_us":431,"threads_started":1}
I20260811 19:17:45.778434 9523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:45.753529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54030 (local address 127.3.61.190:40729)
0811 19:17:45.755731 (+ 2202us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:45.755737 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:45.755818 (+ 81us) server_negotiation.cc:407] Connection header received
0811 19:17:45.758057 (+ 2239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:45.758068 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:45.758467 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:45.758723 (+ 256us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:45.760149 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:45.762047 (+ 1898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:45.763678 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:45.764512 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:45.764708 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:45.769618 (+ 4910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:45.769630 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:45.769682 (+ 52us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:17:45.769723 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:45.773636 (+ 3913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:45.774542 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:45.774552 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:45.774555 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:45.774753 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:45.775228 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:45.775233 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:45.775236 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:45.776022 (+ 786us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:45.776258 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:45.776900 (+ 642us) server_negotiation.cc:299] Negotiation successful
0811 19:17:45.777383 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":140,"server-negotiator.queue_time_us":1971,"spinlock_wait_cycles":29440,"thread_start_us":1625,"threads_started":1}
WARNING: no policy specified for kudu/127.3.61.189@KRBTEST.COM; defaulting to no policy
I20260811 19:17:45.781834 9520 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.61.189" port: 34651 } has no permanent_uuid. Determining permanent_uuid...
I20260811 19:17:45.783011 9522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:45.782504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.189:34651 (local address 127.0.0.1:45326)
0811 19:17:45.782729 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:45.782859 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.189:34651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
W20260811 19:17:45.783299 9402 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.61.189:34651: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.189@KRBTEST.COM" created.
I20260811 19:17:45.785159 9522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:45.784689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.189:34651 (local address 127.0.0.1:45338)
0811 19:17:45.784887 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:45.784988 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.189:34651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260811 19:17:45.785449 9520 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.61.189:34651: Network error: Client connection negotiation failed: client connection to 127.3.61.189:34651: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.189.keytab.
Entry for principal kudu/127.3.61.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.189.keytab.
I20260811 19:17:45.834347 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.189:34651
--webserver_interface=127.3.61.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.189.keytab
--principal=kudu/127.3.61.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:36683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.61.190:40729,127.3.61.189:34651,127.3.61.188:41707
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:17:45.835088 9520 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.61.189" port: 34651 } attempt: 1
I20260811 19:17:45.836495 9522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:45.835914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.189:34651 (local address 127.0.0.1:45348)
0811 19:17:45.836158 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:45.836289 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.189:34651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260811 19:17:45.837769 9522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:45.837219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.189:34651 (local address 127.0.0.1:45354)
0811 19:17:45.837455 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:45.837529 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.189:34651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136}
W20260811 19:17:45.838063 9520 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.61.189:34651: Network error: Client connection negotiation failed: client connection to 127.3.61.189:34651: connect: Connection refused (error 111)
I20260811 19:17:45.906373 9520 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.61.189" port: 34651 } attempt: 2
I20260811 19:17:45.907588 9522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:45.907128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.189:34651 (local address 127.0.0.1:45366)
0811 19:17:45.907350 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:45.907439 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.189:34651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260811 19:17:45.909265 9522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:45.908643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.189:34651 (local address 127.0.0.1:45374)
0811 19:17:45.908897 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:45.909047 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.189:34651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
W20260811 19:17:45.909689 9520 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.61.189:34651: Network error: Client connection negotiation failed: client connection to 127.3.61.189:34651: connect: Connection refused (error 111)
I20260811 19:17:46.001065 9520 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.61.189" port: 34651 } attempt: 3
I20260811 19:17:46.002290 9522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:46.001819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.189:34651 (local address 127.0.0.1:45388)
0811 19:17:46.002035 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:46.002127 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.189:34651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260811 19:17:46.004122 9522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:46.003499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.189:34651 (local address 127.0.0.1:45390)
0811 19:17:46.003750 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:46.003850 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.189:34651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
W20260811 19:17:46.004375 9520 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.61.189:34651: Network error: Client connection negotiation failed: client connection to 127.3.61.189:34651: connect: Connection refused (error 111)
I20260811 19:17:46.147727 9520 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.61.189" port: 34651 } attempt: 4
I20260811 19:17:46.149591 9522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:46.148772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.189:34651 (local address 127.0.0.1:45404)
0811 19:17:46.149237 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:46.149376 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.189:34651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248}
I20260811 19:17:46.151270 9522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:46.150707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.189:34651 (local address 127.0.0.1:45412)
0811 19:17:46.150945 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:46.151077 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.189:34651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
W20260811 19:17:46.151576 9520 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.61.189:34651: Network error: Client connection negotiation failed: client connection to 127.3.61.189:34651: connect: Connection refused (error 111)
W20260811 19:17:46.250499 9530 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 19:17:46.251091 9530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:17:46.251379 9530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:17:46.251497 9530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:17:46.260284 9530 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:17:46.260419 9530 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:17:46.260488 9530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:17:46.260562 9530 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:17:46.260649 9530 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:17:46.272369 9530 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:36683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.3.61.190:40729,127.3.61.189:34651,127.3.61.188:41707
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.189.keytab
--principal=kudu/127.3.61.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.189:34651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.3.61.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:17:46.274155 9530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:17:46.275681 9530 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:17:46.289081 9547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:17:46.288904 9542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:17:46.288866 9544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:17:46.290604 9530 server_base.cc:1034] running on GCE node
I20260811 19:17:46.291797 9530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:17:46.293975 9530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:17:46.295311 9530 hybrid_clock.cc:648] HybridClock initialized: now 1786475866295217 us; error 60 us; skew 500 ppm
I20260811 19:17:46.295760 9530 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:17:46 dist-test-slave-t7g5 krb5kdc[9350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475866, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:17:46.299618 9530 init.cc:377] Logged in from keytab as kudu/127.3.61.189@KRBTEST.COM (short username kudu)
I20260811 19:17:46.301450 9530 webserver.cc:460] Webserver started at http://127.3.61.189:44401/ using document root <none> and password file <none>
I20260811 19:17:46.302294 9530 fs_manager.cc:359] Metadata directory not provided
I20260811 19:17:46.302417 9530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:17:46.302769 9530 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:17:46.306149 9530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/minicluster-data/master-1/data/instance:
uuid: "f9842102c66b4877b4b0a41200e58627"
format_stamp: "Formatted at 2026-08-11 19:17:46 on dist-test-slave-t7g5"
server_key: "bbc093fb8a562f7bd923e49ed72bfbd6"
server_key_iv: "9ba55e8b91d1c4cee24c567fad679a29"
server_key_version: "encryptionkey@0"
I20260811 19:17:46.307025 9530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/minicluster-data/master-1/wal/instance:
uuid: "f9842102c66b4877b4b0a41200e58627"
format_stamp: "Formatted at 2026-08-11 19:17:46 on dist-test-slave-t7g5"
server_key: "bbc093fb8a562f7bd923e49ed72bfbd6"
server_key_iv: "9ba55e8b91d1c4cee24c567fad679a29"
server_key_version: "encryptionkey@0"
I20260811 19:17:46.313565 9530 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260811 19:17:46.318331 9561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:17:46.320158 9530 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 19:17:46.320384 9530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/minicluster-data/master-1/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/minicluster-data/master-1/wal
uuid: "f9842102c66b4877b4b0a41200e58627"
format_stamp: "Formatted at 2026-08-11 19:17:46 on dist-test-slave-t7g5"
server_key: "bbc093fb8a562f7bd923e49ed72bfbd6"
server_key_iv: "9ba55e8b91d1c4cee24c567fad679a29"
server_key_version: "encryptionkey@0"
I20260811 19:17:46.320631 9530 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:17:46.340637 9530 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:17:46.345497 9530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:17:46.345841 9530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:17:46.370484 9530 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.189:34651
I20260811 19:17:46.370499 9652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.189:34651 every 8 connection(s)
I20260811 19:17:46.372417 9530 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/minicluster-data/master-1/data/info.pb
I20260811 19:17:46.377311 9653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:17:46.379271 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 9530
I20260811 19:17:46.379467 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/minicluster-data/master-1/wal/instance
I20260811 19:17:46.379937 3318 external_mini_cluster.cc:1428] Setting key 91eab9d1a07c0551f309ceb4fd01d1fc
I20260811 19:17:46.384697 9653 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.61.190" port: 40729 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 19:17:46 dist-test-slave-t7g5 krb5kdc[9350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475866, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.189@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:17:46.414113 9656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:46.388323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54046 (local address 127.3.61.190:40729)
0811 19:17:46.389045 (+ 722us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:46.389053 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:46.390591 (+ 1538us) server_negotiation.cc:407] Connection header received
0811 19:17:46.393329 (+ 2738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:46.393335 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:46.393465 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:46.393612 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:46.396037 (+ 2425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:46.397133 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:46.399170 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:46.399790 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:46.399938 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:46.405063 (+ 5125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:46.405075 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:46.405081 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:17:46.405177 (+ 96us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:46.409230 (+ 4053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:46.410689 (+ 1459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:46.410697 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:46.410701 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:46.410935 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:46.411546 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:46.411552 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:46.411555 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:46.412036 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:46.412209 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:46.413259 (+ 1050us) server_negotiation.cc:299] Negotiation successful
0811 19:17:46.413598 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":383,"threads_started":1}
I20260811 19:17:46.415303 9657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:46.389284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:40729 (local address 127.0.0.1:54046)
0811 19:17:46.390330 (+ 1046us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:46.390401 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0811 19:17:46.392879 (+ 2478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:17:46.394115 (+ 1236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:17:46.394129 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:17:46.394969 (+ 840us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:17:46.395705 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:17:46.395731 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:17:46.397489 (+ 1758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:17:46.397494 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:17:46.398956 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:17:46.398976 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:17:46.400111 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:17:46.400117 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:17:46.400373 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:46.401255 (+ 882us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:17:46.401287 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:17:46.404560 (+ 3273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:17:46.409562 (+ 5002us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:17:46.409576 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:17:46.409600 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:17:46.410382 (+ 782us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:17:46.411103 (+ 721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:17:46.411108 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:17:46.411111 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:17:46.411368 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:17:46.412208 (+ 840us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:17:46.412217 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:17:46.412772 (+ 555us) client_negotiation.cc:769] Sending connection context
0811 19:17:46.413088 (+ 316us) client_negotiation.cc:240] Negotiation successful
0811 19:17:46.413781 (+ 693us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":674,"thread_start_us":422,"threads_started":1}
I20260811 19:17:46.418187 9653 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.61.189" port: 34651 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.3.61.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.188@KRBTEST.COM" created.
Aug 11 19:17:46 dist-test-slave-t7g5 krb5kdc[9350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475866, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.189@KRBTEST.COM for kudu/127.3.61.189@KRBTEST.COM
I20260811 19:17:46.437728 9657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:46.419132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.189:34651 (local address 127.0.0.1:45416)
0811 19:17:46.419361 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:46.419384 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 19:17:46.419529 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:17:46.420695 (+ 1166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:17:46.420700 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:17:46.421251 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:17:46.421459 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:17:46.421472 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:17:46.423139 (+ 1667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:17:46.423145 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:17:46.424492 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:17:46.424509 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:17:46.425625 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:17:46.425633 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:17:46.425838 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:46.426479 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:17:46.426503 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:17:46.431389 (+ 4886us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:17:46.435003 (+ 3614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:17:46.435009 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:17:46.435013 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:17:46.435374 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:17:46.435774 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:17:46.435778 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:17:46.435782 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:17:46.435930 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:17:46.436601 (+ 671us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:17:46.436606 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:17:46.436743 (+ 137us) client_negotiation.cc:769] Sending connection context
0811 19:17:46.436875 (+ 132us) client_negotiation.cc:240] Negotiation successful
0811 19:17:46.437189 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90}
I20260811 19:17:46.437762 9661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:46.419473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45416 (local address 127.3.61.189:34651)
0811 19:17:46.420154 (+ 681us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:46.420161 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:46.420192 (+ 31us) server_negotiation.cc:407] Connection header received
0811 19:17:46.420249 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:46.420255 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:46.420479 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:46.420610 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:46.421727 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:46.422964 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:46.424700 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:46.425436 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:46.425551 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:46.431699 (+ 6148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:46.431719 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:46.431740 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:17:46.431777 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:46.434832 (+ 3055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:46.435524 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:46.435532 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:46.435534 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:46.435657 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:46.436038 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:46.436041 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:46.436043 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:46.436449 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:46.436664 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:46.437061 (+ 397us) server_negotiation.cc:299] Negotiation successful
0811 19:17:46.437283 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":511,"thread_start_us":371,"threads_started":1}
I20260811 19:17:46.440846 9653 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.61.188" port: 41707 } has no permanent_uuid. Determining permanent_uuid...
I20260811 19:17:46.442227 9657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:46.441724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:41707 (local address 127.0.0.1:43554)
0811 19:17:46.441917 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:46.442060 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.188:41707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 19:17:46.442581 9565 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.61.188:41707: connect: Connection refused (error 111)
I20260811 19:17:46.444707 9657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:46.444123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:41707 (local address 127.0.0.1:43570)
0811 19:17:46.444377 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:46.444528 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.188:41707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
W20260811 19:17:46.445051 9653 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.61.188:41707: Network error: Client connection negotiation failed: client connection to 127.3.61.188:41707: connect: Connection refused (error 111)
I20260811 19:17:46.450891 9520 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.61.189" port: 34651 } attempt: 5
Aug 11 19:17:46 dist-test-slave-t7g5 krb5kdc[9350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475865, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.190@KRBTEST.COM for kudu/127.3.61.189@KRBTEST.COM
I20260811 19:17:46.467603 9661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:46.452242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45424 (local address 127.3.61.189:34651)
0811 19:17:46.452443 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:46.452449 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:46.452475 (+ 26us) server_negotiation.cc:407] Connection header received
0811 19:17:46.452684 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:46.452691 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:46.452823 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:46.452959 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:46.454018 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:46.454755 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:46.456374 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:46.456819 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:46.456924 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:46.460901 (+ 3977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:46.460908 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:46.460912 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:17:46.460950 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:46.463711 (+ 2761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:46.464653 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:46.464661 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:46.464664 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:46.464937 (+ 273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:46.465556 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:46.465562 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:46.465564 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:46.466100 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:46.466282 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:46.467052 (+ 770us) server_negotiation.cc:299] Negotiation successful
0811 19:17:46.467332 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 19:17:46.467653 9522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:46.451915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.189:34651 (local address 127.0.0.1:45424)
0811 19:17:46.452187 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:46.452213 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 19:17:46.452388 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:17:46.453018 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:17:46.453023 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:17:46.453526 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:17:46.453813 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:17:46.453831 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:17:46.454903 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:17:46.454909 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:17:46.456158 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:17:46.456174 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:17:46.456989 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:17:46.456995 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:17:46.457178 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:46.457976 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:17:46.457997 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:17:46.460623 (+ 2626us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:17:46.463907 (+ 3284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:17:46.463912 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:17:46.463915 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:17:46.464396 (+ 481us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:17:46.465138 (+ 742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:17:46.465144 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:17:46.465148 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:17:46.465362 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:17:46.466298 (+ 936us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:17:46.466304 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:17:46.466562 (+ 258us) client_negotiation.cc:769] Sending connection context
0811 19:17:46.466820 (+ 258us) client_negotiation.cc:240] Negotiation successful
0811 19:17:46.467190 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95}
I20260811 19:17:46.469123 9520 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.61.188" port: 41707 } has no permanent_uuid. Determining permanent_uuid...
I20260811 19:17:46.470582 9522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:46.470071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:41707 (local address 127.0.0.1:43584)
0811 19:17:46.470333 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:46.470423 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.188:41707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133}
I20260811 19:17:46.471776 9522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:46.471409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:41707 (local address 127.0.0.1:43590)
0811 19:17:46.471565 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:46.471635 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.188:41707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260811 19:17:46.472008 9520 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.61.188:41707: Network error: Client connection negotiation failed: client connection to 127.3.61.188:41707: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.188.keytab.
Entry for principal kudu/127.3.61.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.188.keytab.
W20260811 19:17:46.480854 3318 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.3.61.188: real 0.101s user 0.000s sys 0.041s
I20260811 19:17:46.481701 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.188:41707
--webserver_interface=127.3.61.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.188.keytab
--principal=kudu/127.3.61.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:36683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.61.190:40729,127.3.61.189:34651,127.3.61.188:41707
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:17:46.494488 9653 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.61.188" port: 41707 } attempt: 1
I20260811 19:17:46.495697 9657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:46.495215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:41707 (local address 127.0.0.1:43594)
0811 19:17:46.495425 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:46.495531 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.188:41707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260811 19:17:46.496933 9657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:46.496532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:41707 (local address 127.0.0.1:43604)
0811 19:17:46.496699 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:46.496792 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.188:41707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260811 19:17:46.497189 9653 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.61.188:41707: Network error: Client connection negotiation failed: client connection to 127.3.61.188:41707: connect: Connection refused (error 111)
I20260811 19:17:46.523339 9520 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.61.188" port: 41707 } attempt: 1
I20260811 19:17:46.525137 9522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:46.524330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:41707 (local address 127.0.0.1:43616)
0811 19:17:46.524711 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:46.524853 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.188:41707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260811 19:17:46.526487 9522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:46.526109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:41707 (local address 127.0.0.1:43630)
0811 19:17:46.526275 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:46.526359 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.188:41707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260811 19:17:46.526728 9520 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.61.188:41707: Network error: Client connection negotiation failed: client connection to 127.3.61.188:41707: connect: Connection refused (error 111)
I20260811 19:17:46.565466 9653 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.61.188" port: 41707 } attempt: 2
I20260811 19:17:46.567211 9657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:46.566486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:41707 (local address 127.0.0.1:43636)
0811 19:17:46.566808 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:46.566982 (+ 174us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.188:41707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260811 19:17:46.568670 9657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:46.568215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:41707 (local address 127.0.0.1:43644)
0811 19:17:46.568386 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:46.568484 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.188:41707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260811 19:17:46.569094 9653 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.61.188:41707: Network error: Client connection negotiation failed: client connection to 127.3.61.188:41707: connect: Connection refused (error 111)
I20260811 19:17:46.594987 9520 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.61.188" port: 41707 } attempt: 2
I20260811 19:17:46.596509 9522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:46.595954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:41707 (local address 127.0.0.1:43654)
0811 19:17:46.596216 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:46.596331 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.188:41707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260811 19:17:46.597486 9522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:46.597151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:41707 (local address 127.0.0.1:43664)
0811 19:17:46.597300 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:46.597367 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.188:41707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 19:17:46.597720 9520 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.61.188:41707: Network error: Client connection negotiation failed: client connection to 127.3.61.188:41707: connect: Connection refused (error 111)
I20260811 19:17:46.660494 9653 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.61.188" port: 41707 } attempt: 3
I20260811 19:17:46.662278 9657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:46.661340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:41707 (local address 127.0.0.1:43676)
0811 19:17:46.661797 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:46.661967 (+ 170us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.188:41707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156}
I20260811 19:17:46.663817 9657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:46.663247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:41707 (local address 127.0.0.1:43686)
0811 19:17:46.663573 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:46.663674 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.188:41707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207}
W20260811 19:17:46.664113 9653 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.61.188:41707: Network error: Client connection negotiation failed: client connection to 127.3.61.188:41707: connect: Connection refused (error 111)
I20260811 19:17:46.704044 9520 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.61.188" port: 41707 } attempt: 3
I20260811 19:17:46.705825 9522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:46.704850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:41707 (local address 127.0.0.1:43702)
0811 19:17:46.705271 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:46.705460 (+ 189us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.188:41707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148}
I20260811 19:17:46.707152 9522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:46.706688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:41707 (local address 127.0.0.1:43712)
0811 19:17:46.706933 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:46.707023 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.188:41707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
W20260811 19:17:46.707397 9520 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.61.188:41707: Network error: Client connection negotiation failed: client connection to 127.3.61.188:41707: connect: Connection refused (error 111)
I20260811 19:17:46.807408 9653 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.61.188" port: 41707 } attempt: 4
I20260811 19:17:46.808916 9657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:46.808202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:41707 (local address 127.0.0.1:43718)
0811 19:17:46.808554 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:46.808692 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.188:41707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260811 19:17:46.810386 9657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:46.809935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:41707 (local address 127.0.0.1:43732)
0811 19:17:46.810127 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:46.810217 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.188:41707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260811 19:17:46.810678 9653 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.61.188:41707: Network error: Client connection negotiation failed: client connection to 127.3.61.188:41707: connect: Connection refused (error 111)
W20260811 19:17:46.880151 9668 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 19:17:46.880529 9668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:17:46.880630 9668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:17:46.880754 9668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 19:17:46.884661 9520 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.61.188" port: 41707 } attempt: 4
I20260811 19:17:46.885903 9522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:46.885376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:41707 (local address 127.0.0.1:43740)
0811 19:17:46.885593 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:46.885732 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.188:41707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260811 19:17:46.887001 9522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:46.886655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:41707 (local address 127.0.0.1:43746)
0811 19:17:46.886806 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:46.886881 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.188:41707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260811 19:17:46.887228 9520 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.61.188:41707: Network error: Client connection negotiation failed: client connection to 127.3.61.188:41707: connect: Connection refused (error 111)
W20260811 19:17:46.889811 9668 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:17:46.889961 9668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:17:46.890030 9668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:17:46.890081 9668 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:17:46.890205 9668 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:17:46.903585 9668 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:36683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.3.61.190:40729,127.3.61.189:34651,127.3.61.188:41707
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.188.keytab
--principal=kudu/127.3.61.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.188:41707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.3.61.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:17:46.905586 9668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:17:46.907168 9668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:17:46.920075 9679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:17:46.920300 9678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:17:46.920277 9686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:17:46.921622 9668 server_base.cc:1034] running on GCE node
I20260811 19:17:46.922555 9668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:17:46.924865 9668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:17:46.926122 9668 hybrid_clock.cc:648] HybridClock initialized: now 1786475866926055 us; error 50 us; skew 500 ppm
I20260811 19:17:46.926506 9668 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:17:46 dist-test-slave-t7g5 krb5kdc[9350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475866, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:17:46.930038 9668 init.cc:377] Logged in from keytab as kudu/127.3.61.188@KRBTEST.COM (short username kudu)
I20260811 19:17:46.931803 9668 webserver.cc:460] Webserver started at http://127.3.61.188:39601/ using document root <none> and password file <none>
I20260811 19:17:46.932544 9668 fs_manager.cc:359] Metadata directory not provided
I20260811 19:17:46.932655 9668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:17:46.933012 9668 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:17:46.936363 9668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/minicluster-data/master-2/data/instance:
uuid: "ef9d5ff698a04c928be5f31171ed1b5f"
format_stamp: "Formatted at 2026-08-11 19:17:46 on dist-test-slave-t7g5"
server_key: "750464b83369c035645cf5a7344fc8fa"
server_key_iv: "0c83aee26f21373f047f19475f22fcbf"
server_key_version: "encryptionkey@0"
I20260811 19:17:46.937206 9668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/minicluster-data/master-2/wal/instance:
uuid: "ef9d5ff698a04c928be5f31171ed1b5f"
format_stamp: "Formatted at 2026-08-11 19:17:46 on dist-test-slave-t7g5"
server_key: "750464b83369c035645cf5a7344fc8fa"
server_key_iv: "0c83aee26f21373f047f19475f22fcbf"
server_key_version: "encryptionkey@0"
I20260811 19:17:46.943059 9668 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260811 19:17:46.946851 9697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:17:46.948589 9668 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 19:17:46.948791 9668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/minicluster-data/master-2/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/minicluster-data/master-2/wal
uuid: "ef9d5ff698a04c928be5f31171ed1b5f"
format_stamp: "Formatted at 2026-08-11 19:17:46 on dist-test-slave-t7g5"
server_key: "750464b83369c035645cf5a7344fc8fa"
server_key_iv: "0c83aee26f21373f047f19475f22fcbf"
server_key_version: "encryptionkey@0"
I20260811 19:17:46.949040 9668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:17:46.974083 9668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:17:46.979902 9668 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:17:46.980222 9668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:17:47.002435 9668 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.188:41707
I20260811 19:17:47.002475 9801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.188:41707 every 8 connection(s)
I20260811 19:17:47.004040 9668 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/minicluster-data/master-2/data/info.pb
I20260811 19:17:47.008131 9802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:17:47.013386 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 9668
I20260811 19:17:47.013564 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786475454804547-3318-0/minicluster-data/master-2/wal/instance
I20260811 19:17:47.014070 3318 external_mini_cluster.cc:1428] Setting key 5f2e4e921943ea1f4e76df8d1e65e2d0
I20260811 19:17:47.015623 9802 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.61.190" port: 40729 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 19:17:47 dist-test-slave-t7g5 krb5kdc[9350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475866, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.188@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
Aug 11 19:17:47 dist-test-slave-t7g5 krb5kdc[9350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.188@KRBTEST.COM
I20260811 19:17:47.039642 9811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:47.018644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54062 (local address 127.3.61.190:40729)
0811 19:17:47.020064 (+ 1420us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:47.020070 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:47.020092 (+ 22us) server_negotiation.cc:407] Connection header received
0811 19:17:47.020158 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:47.020162 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:47.020254 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:47.020359 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:47.022172 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:47.023309 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:47.025640 (+ 2331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:47.026344 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:47.026492 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:47.031337 (+ 4845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:47.031348 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:47.031354 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:17:47.031439 (+ 85us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:47.035117 (+ 3678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:47.036113 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:47.036119 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:47.036122 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:47.036313 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:47.036877 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:47.036883 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:47.036886 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:47.037231 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:47.037421 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:47.038902 (+ 1481us) server_negotiation.cc:299] Negotiation successful
0811 19:17:47.039180 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1299,"thread_start_us":1206,"threads_started":1}
I20260811 19:17:47.041595 9809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:47.018546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:40729 (local address 127.0.0.1:54062)
0811 19:17:47.019465 (+ 919us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:47.019524 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 19:17:47.019934 (+ 410us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:17:47.020446 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:17:47.020452 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:17:47.021366 (+ 914us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:17:47.021940 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:17:47.021969 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:17:47.023539 (+ 1570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:17:47.023545 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:17:47.025388 (+ 1843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:17:47.025406 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:17:47.026645 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:17:47.026654 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:17:47.026833 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:47.027697 (+ 864us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:17:47.027728 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:17:47.030932 (+ 3204us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:17:47.035304 (+ 4372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:17:47.035319 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:17:47.035335 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:17:47.035966 (+ 631us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:17:47.036514 (+ 548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:17:47.036518 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:17:47.036521 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:17:47.036734 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:17:47.037547 (+ 813us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:17:47.037561 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:17:47.038361 (+ 800us) client_negotiation.cc:769] Sending connection context
0811 19:17:47.038760 (+ 399us) client_negotiation.cc:240] Negotiation successful
0811 19:17:47.039466 (+ 706us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"spinlock_wait_cycles":29952,"thread_start_us":403,"threads_started":1}
I20260811 19:17:47.041685 9807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:47.016431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43758 (local address 127.3.61.188:41707)
0811 19:17:47.017242 (+ 811us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:47.017254 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:47.017300 (+ 46us) server_negotiation.cc:407] Connection header received
0811 19:17:47.019764 (+ 2464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:47.019798 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:47.020132 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:47.020371 (+ 239us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:47.022325 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:47.023907 (+ 1582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:47.025707 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:47.026604 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:47.026784 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:47.032226 (+ 5442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:47.032240 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:47.032262 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:17:47.032313 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:47.036032 (+ 3719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:47.036902 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:47.036909 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:47.036911 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:47.037042 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:47.037547 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:47.037555 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:47.037558 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:47.038324 (+ 766us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:47.038775 (+ 451us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:47.039179 (+ 404us) server_negotiation.cc:299] Negotiation successful
0811 19:17:47.039616 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":156,"server-negotiator.queue_time_us":593,"spinlock_wait_cycles":19840,"thread_start_us":371,"threads_started":1}
I20260811 19:17:47.043944 9802 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.61.189" port: 34651 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 19:17:47 dist-test-slave-t7g5 krb5kdc[9350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475866, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.188@KRBTEST.COM for kudu/127.3.61.189@KRBTEST.COM
Aug 11 19:17:47 dist-test-slave-t7g5 krb5kdc[9350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:17:47.061551 9809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:47.044685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.189:34651 (local address 127.0.0.1:45426)
0811 19:17:47.044868 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:47.044889 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 19:17:47.045009 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:17:47.045631 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:17:47.045637 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:17:47.046182 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:17:47.046398 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:17:47.046412 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:17:47.047747 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:17:47.047754 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:17:47.048939 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:17:47.048956 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:17:47.049759 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:17:47.049765 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:17:47.049974 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:47.050660 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:17:47.050685 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:17:47.053614 (+ 2929us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:17:47.058668 (+ 5054us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:17:47.058675 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:17:47.058679 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:17:47.059212 (+ 533us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:17:47.059755 (+ 543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:17:47.059759 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:17:47.059761 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:17:47.059924 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:17:47.060531 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:17:47.060536 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:17:47.060703 (+ 167us) client_negotiation.cc:769] Sending connection context
0811 19:17:47.060827 (+ 124us) client_negotiation.cc:240] Negotiation successful
0811 19:17:47.061161 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 19:17:47.061684 9813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:47.044810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45426 (local address 127.3.61.189:34651)
0811 19:17:47.045289 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:47.045295 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:47.045314 (+ 19us) server_negotiation.cc:407] Connection header received
0811 19:17:47.045379 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:47.045383 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:47.045460 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:47.045579 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:47.046630 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:47.047369 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:47.049120 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:47.049565 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:47.049716 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:47.053930 (+ 4214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:47.053938 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:47.053943 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:17:47.053991 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:47.058463 (+ 4472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:47.059353 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:47.059359 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:47.059363 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:47.059556 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:47.060070 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:47.060076 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:47.060080 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:47.060392 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:47.060557 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:47.060972 (+ 415us) server_negotiation.cc:299] Negotiation successful
0811 19:17:47.061236 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":260,"threads_started":1}
I20260811 19:17:47.062899 9802 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.61.188" port: 41707 } has no permanent_uuid. Determining permanent_uuid...
I20260811 19:17:47.064939 9811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:47.045383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54066 (local address 127.3.61.190:40729)
0811 19:17:47.045576 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:47.045581 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:47.045600 (+ 19us) server_negotiation.cc:407] Connection header received
0811 19:17:47.045738 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:47.045743 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:47.045820 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:47.045900 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:47.047895 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:47.049002 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:47.050687 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:47.051154 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:47.051257 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:47.057910 (+ 6653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:47.057922 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:47.057928 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:17:47.058002 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:47.061196 (+ 3194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:47.062147 (+ 951us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:47.062152 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:47.062155 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:47.062301 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:47.062740 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:47.062744 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:47.062746 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:47.063135 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:47.063447 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:47.064323 (+ 876us) server_negotiation.cc:299] Negotiation successful
0811 19:17:47.064606 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
Aug 11 19:17:47 dist-test-slave-t7g5 krb5kdc[9350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475866, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.188@KRBTEST.COM for kudu/127.3.61.188@KRBTEST.COM
Aug 11 19:17:47 dist-test-slave-t7g5 krb5kdc[9350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.189@KRBTEST.COM
I20260811 19:17:47.080169 9809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:47.064064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:41707 (local address 127.0.0.1:43764)
0811 19:17:47.064536 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:47.064555 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 19:17:47.064679 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:17:47.065045 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:17:47.065050 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:17:47.065567 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:17:47.065769 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:17:47.065779 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:17:47.067210 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:17:47.067217 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:17:47.068616 (+ 1399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:17:47.068634 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:17:47.069328 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:17:47.069334 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:17:47.069544 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:47.070199 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:17:47.070221 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:17:47.073493 (+ 3272us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:17:47.076970 (+ 3477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:17:47.076976 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:17:47.076979 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:17:47.077453 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:17:47.078006 (+ 553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:17:47.078011 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:17:47.078015 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:17:47.078218 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:17:47.078958 (+ 740us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:17:47.078965 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:17:47.079247 (+ 282us) client_negotiation.cc:769] Sending connection context
0811 19:17:47.079458 (+ 211us) client_negotiation.cc:240] Negotiation successful
0811 19:17:47.079826 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364}
I20260811 19:17:47.080662 9807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:47.064021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43764 (local address 127.3.61.188:41707)
0811 19:17:47.064225 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:47.064231 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:47.064646 (+ 415us) server_negotiation.cc:407] Connection header received
0811 19:17:47.064822 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:47.064827 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:47.064890 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:47.064984 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:47.065984 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:47.066899 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:47.068768 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:47.069176 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:47.069304 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:47.073879 (+ 4575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:47.073887 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:47.073892 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:17:47.073935 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:47.076736 (+ 2801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:47.077628 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:47.077633 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:47.077637 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:47.077870 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:47.078400 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:47.078407 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:47.078409 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:47.078786 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:47.078945 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:47.079580 (+ 635us) server_negotiation.cc:299] Negotiation successful
0811 19:17:47.080066 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":166,"server-negotiator.queue_time_us":71}
I20260811 19:17:47.085246 9813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:47.067164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45430 (local address 127.3.61.189:34651)
0811 19:17:47.067447 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:47.067454 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:47.067520 (+ 66us) server_negotiation.cc:407] Connection header received
0811 19:17:47.067594 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:47.067601 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:47.067707 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:47.067836 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:47.069166 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:47.070026 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:47.071828 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:47.072742 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:47.072859 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:47.077970 (+ 5111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:47.077978 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:47.077983 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:17:47.078032 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:47.081511 (+ 3479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:47.082557 (+ 1046us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:47.082563 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:47.082566 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:47.082765 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:47.083422 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:47.083427 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:47.083431 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:47.083851 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:47.084136 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:47.084587 (+ 451us) server_negotiation.cc:299] Negotiation successful
0811 19:17:47.084876 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260811 19:17:47.087189 9802 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef9d5ff698a04c928be5f31171ed1b5f: Bootstrap starting.
I20260811 19:17:47.091347 9802 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef9d5ff698a04c928be5f31171ed1b5f: Neither blocks nor log segments found. Creating new log.
I20260811 19:17:47.092507 9802 log.cc:824] T 00000000000000000000000000000000 P ef9d5ff698a04c928be5f31171ed1b5f: Log is configured to *not* fsync() on all Append() calls
I20260811 19:17:47.092655 3318 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:17:47.096123 9802 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef9d5ff698a04c928be5f31171ed1b5f: No bootstrap required, opened a new log
I20260811 19:17:47.096254 9468 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "fc2444314f864d188196194f1cc75c46"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54066
I20260811 19:17:47.104358 9802 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef9d5ff698a04c928be5f31171ed1b5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc2444314f864d188196194f1cc75c46" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 40729 } } peers { permanent_uuid: "f9842102c66b4877b4b0a41200e58627" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 34651 } } peers { permanent_uuid: "ef9d5ff698a04c928be5f31171ed1b5f" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 41707 } }
I20260811 19:17:47.104804 9802 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef9d5ff698a04c928be5f31171ed1b5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef9d5ff698a04c928be5f31171ed1b5f, State: Initialized, Role: FOLLOWER
I20260811 19:17:47.105413 9802 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef9d5ff698a04c928be5f31171ed1b5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc2444314f864d188196194f1cc75c46" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 40729 } } peers { permanent_uuid: "f9842102c66b4877b4b0a41200e58627" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 34651 } } peers { permanent_uuid: "ef9d5ff698a04c928be5f31171ed1b5f" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 41707 } }
I20260811 19:17:47.106913 9816 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef9d5ff698a04c928be5f31171ed1b5f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc2444314f864d188196194f1cc75c46" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 40729 } } peers { permanent_uuid: "f9842102c66b4877b4b0a41200e58627" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 34651 } } peers { permanent_uuid: "ef9d5ff698a04c928be5f31171ed1b5f" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 41707 } } }
I20260811 19:17:47.107404 9816 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef9d5ff698a04c928be5f31171ed1b5f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:17:47.108188 9802 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef9d5ff698a04c928be5f31171ed1b5f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:17:47.109941 9653 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.61.188" port: 41707 } attempt: 5
Aug 11 19:17:47 dist-test-slave-t7g5 krb5kdc[9350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475866, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.189@KRBTEST.COM for kudu/127.3.61.188@KRBTEST.COM
W20260811 19:17:47.124774 9835 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ef9d5ff698a04c928be5f31171ed1b5f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 19:17:47.124989 9835 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 19:17:47.128165 9657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:47.110880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:41707 (local address 127.0.0.1:43776)
0811 19:17:47.111165 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:47.111187 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 19:17:47.111338 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:17:47.111860 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:17:47.111867 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:17:47.112421 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:17:47.112643 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:17:47.112654 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:17:47.114536 (+ 1882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:17:47.114542 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:17:47.115477 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:17:47.115492 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:17:47.116351 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:17:47.116366 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:17:47.116596 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:47.117200 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:17:47.117220 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:17:47.119929 (+ 2709us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:17:47.124842 (+ 4913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:17:47.124849 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:17:47.124852 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:17:47.125454 (+ 602us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:17:47.126017 (+ 563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:17:47.126022 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:17:47.126026 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:17:47.126206 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:17:47.127097 (+ 891us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:17:47.127102 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:17:47.127266 (+ 164us) client_negotiation.cc:769] Sending connection context
0811 19:17:47.127421 (+ 155us) client_negotiation.cc:240] Negotiation successful
0811 19:17:47.127747 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98}
I20260811 19:17:47.129221 9807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:47.111216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43776 (local address 127.3.61.188:41707)
0811 19:17:47.111410 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:47.111417 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:47.111445 (+ 28us) server_negotiation.cc:407] Connection header received
0811 19:17:47.111518 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:47.111524 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:47.111612 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:47.111755 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:47.112861 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:47.114235 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:47.115653 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:47.116187 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:47.116307 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:47.120202 (+ 3895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:47.120209 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:47.120214 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:17:47.120248 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:47.124307 (+ 4059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:47.125600 (+ 1293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:47.125606 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:47.125609 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:47.125800 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:47.126343 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:47.126348 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:47.126351 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:47.126865 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:47.127017 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:47.128567 (+ 1550us) server_negotiation.cc:299] Negotiation successful
0811 19:17:47.128838 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 19:17:47.121840 9668 master_runner.cc:186] Error getting master registration for 127.3.61.190:40729: OK, instance_id { permanent_uuid: "fc2444314f864d188196194f1cc75c46" instance_seqno: 1786475865727804 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 19:17:47.132622 9668 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:17:47.135838 9653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9842102c66b4877b4b0a41200e58627: Bootstrap starting.
I20260811 19:17:47.140256 9653 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f9842102c66b4877b4b0a41200e58627: Neither blocks nor log segments found. Creating new log.
I20260811 19:17:47.141636 9653 log.cc:824] T 00000000000000000000000000000000 P f9842102c66b4877b4b0a41200e58627: Log is configured to *not* fsync() on all Append() calls
I20260811 19:17:47.145612 9653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9842102c66b4877b4b0a41200e58627: No bootstrap required, opened a new log
I20260811 19:17:47.157945 9653 raft_consensus.cc:348] T 00000000000000000000000000000000 P f9842102c66b4877b4b0a41200e58627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc2444314f864d188196194f1cc75c46" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 40729 } } peers { permanent_uuid: "f9842102c66b4877b4b0a41200e58627" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 34651 } } peers { permanent_uuid: "ef9d5ff698a04c928be5f31171ed1b5f" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 41707 } }
I20260811 19:17:47.158468 9653 raft_consensus.cc:729] T 00000000000000000000000000000000 P f9842102c66b4877b4b0a41200e58627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9842102c66b4877b4b0a41200e58627, State: Initialized, Role: FOLLOWER
I20260811 19:17:47.159106 9653 consensus_queue.cc:260] T 00000000000000000000000000000000 P f9842102c66b4877b4b0a41200e58627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc2444314f864d188196194f1cc75c46" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 40729 } } peers { permanent_uuid: "f9842102c66b4877b4b0a41200e58627" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 34651 } } peers { permanent_uuid: "ef9d5ff698a04c928be5f31171ed1b5f" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 41707 } }
I20260811 19:17:47.160259 9842 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9842102c66b4877b4b0a41200e58627 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc2444314f864d188196194f1cc75c46" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 40729 } } peers { permanent_uuid: "f9842102c66b4877b4b0a41200e58627" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 34651 } } peers { permanent_uuid: "ef9d5ff698a04c928be5f31171ed1b5f" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 41707 } } }
I20260811 19:17:47.160687 9842 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9842102c66b4877b4b0a41200e58627 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:17:47.161746 9653 sys_catalog.cc:564] T 00000000000000000000000000000000 P f9842102c66b4877b4b0a41200e58627 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:17:47.164446 9520 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.61.188" port: 41707 } attempt: 5
Aug 11 19:17:47 dist-test-slave-t7g5 krb5kdc[9350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475865, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.190@KRBTEST.COM for kudu/127.3.61.188@KRBTEST.COM
I20260811 19:17:47.169772 9530 master_runner.cc:186] Error getting master registration for 127.3.61.190:40729: OK, instance_id { permanent_uuid: "fc2444314f864d188196194f1cc75c46" instance_seqno: 1786475865727804 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 19:17:47.182337 9530 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 19:17:47.185626 9871 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f9842102c66b4877b4b0a41200e58627: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 19:17:47.185906 9871 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 19:17:47.187660 9807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:47.165268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43792 (local address 127.3.61.188:41707)
0811 19:17:47.165454 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:47.165459 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:47.165547 (+ 88us) server_negotiation.cc:407] Connection header received
0811 19:17:47.165796 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:47.165803 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:47.165928 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:47.166041 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:47.167185 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:47.168294 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:47.170026 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:47.170771 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:47.170918 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:47.176498 (+ 5580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:47.176509 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:47.176516 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:17:47.176598 (+ 82us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:47.183679 (+ 7081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:47.184891 (+ 1212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:47.184897 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:47.184900 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:47.185211 (+ 311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:47.185963 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:47.185969 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:47.185972 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:47.186448 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:47.186599 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:47.187039 (+ 440us) server_negotiation.cc:299] Negotiation successful
0811 19:17:47.187305 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 19:17:47.187662 9522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:47.165174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:41707 (local address 127.0.0.1:43792)
0811 19:17:47.165433 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:47.165458 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 19:17:47.165622 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:17:47.166085 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:17:47.166092 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:17:47.166678 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:17:47.166918 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:17:47.166933 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:17:47.168443 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:17:47.168449 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:17:47.169850 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:17:47.169885 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:17:47.170930 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:17:47.170937 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:17:47.171152 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:47.171988 (+ 836us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:17:47.172012 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:17:47.175955 (+ 3943us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:17:47.184004 (+ 8049us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:17:47.184011 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:17:47.184016 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:17:47.184608 (+ 592us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:17:47.185413 (+ 805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:17:47.185418 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:17:47.185421 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:17:47.185718 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:17:47.186570 (+ 852us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:17:47.186575 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:17:47.186768 (+ 193us) client_negotiation.cc:769] Sending connection context
0811 19:17:47.186918 (+ 150us) client_negotiation.cc:240] Negotiation successful
0811 19:17:47.187240 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 19:17:47.194535 9520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc2444314f864d188196194f1cc75c46: Bootstrap starting.
I20260811 19:17:47.199100 9520 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc2444314f864d188196194f1cc75c46: Neither blocks nor log segments found. Creating new log.
I20260811 19:17:47.200627 9520 log.cc:824] T 00000000000000000000000000000000 P fc2444314f864d188196194f1cc75c46: Log is configured to *not* fsync() on all Append() calls
I20260811 19:17:47.206983 9520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc2444314f864d188196194f1cc75c46: No bootstrap required, opened a new log
I20260811 19:17:47.223894 9520 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc2444314f864d188196194f1cc75c46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc2444314f864d188196194f1cc75c46" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 40729 } } peers { permanent_uuid: "f9842102c66b4877b4b0a41200e58627" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 34651 } } peers { permanent_uuid: "ef9d5ff698a04c928be5f31171ed1b5f" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 41707 } }
I20260811 19:17:47.224715 9520 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc2444314f864d188196194f1cc75c46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc2444314f864d188196194f1cc75c46, State: Initialized, Role: FOLLOWER
I20260811 19:17:47.225740 9520 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc2444314f864d188196194f1cc75c46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc2444314f864d188196194f1cc75c46" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 40729 } } peers { permanent_uuid: "f9842102c66b4877b4b0a41200e58627" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 34651 } } peers { permanent_uuid: "ef9d5ff698a04c928be5f31171ed1b5f" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 41707 } }
I20260811 19:17:47.227547 9877 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc2444314f864d188196194f1cc75c46 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc2444314f864d188196194f1cc75c46" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 40729 } } peers { permanent_uuid: "f9842102c66b4877b4b0a41200e58627" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 34651 } } peers { permanent_uuid: "ef9d5ff698a04c928be5f31171ed1b5f" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 41707 } } }
I20260811 19:17:47.228276 9877 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc2444314f864d188196194f1cc75c46 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:17:47.230628 9468 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc2444314f864d188196194f1cc75c46 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 19:17:47.230918 9468 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc2444314f864d188196194f1cc75c46 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:17:47.231181 9520 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc2444314f864d188196194f1cc75c46 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 19:17:47.232664 9712 tablet.cc:2369] T 00000000000000000000000000000000 P ef9d5ff698a04c928be5f31171ed1b5f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:17:47.235258 9468 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc2444314f864d188196194f1cc75c46 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc2444314f864d188196194f1cc75c46" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 40729 } } peers { permanent_uuid: "f9842102c66b4877b4b0a41200e58627" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 34651 } } peers { permanent_uuid: "ef9d5ff698a04c928be5f31171ed1b5f" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 41707 } }
I20260811 19:17:47.238417 9603 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fc2444314f864d188196194f1cc75c46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f9842102c66b4877b4b0a41200e58627"
I20260811 19:17:47.238997 9603 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f9842102c66b4877b4b0a41200e58627 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:17:47.243364 9603 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f9842102c66b4877b4b0a41200e58627 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc2444314f864d188196194f1cc75c46 in term 1.
I20260811 19:17:47.244781 9402 leader_election.cc:302] T 00000000000000000000000000000000 P fc2444314f864d188196194f1cc75c46 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f9842102c66b4877b4b0a41200e58627, fc2444314f864d188196194f1cc75c46; no voters:
I20260811 19:17:47.245846 9877 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc2444314f864d188196194f1cc75c46 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:17:47.251174 9753 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fc2444314f864d188196194f1cc75c46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "ef9d5ff698a04c928be5f31171ed1b5f"
I20260811 19:17:47.251955 9753 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef9d5ff698a04c928be5f31171ed1b5f [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:17:47.257411 9753 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ef9d5ff698a04c928be5f31171ed1b5f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc2444314f864d188196194f1cc75c46 in term 1.
I20260811 19:17:47.258852 9468 leader_election.cc:288] T 00000000000000000000000000000000 P fc2444314f864d188196194f1cc75c46 [CANDIDATE]: Term 1 election: Requested vote from peers f9842102c66b4877b4b0a41200e58627 (127.3.61.189:34651), ef9d5ff698a04c928be5f31171ed1b5f (127.3.61.188:41707)
I20260811 19:17:47.278618 9877 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc2444314f864d188196194f1cc75c46 [term 1 LEADER]: Becoming Leader. State: Replica: fc2444314f864d188196194f1cc75c46, State: Running, Role: LEADER
I20260811 19:17:47.279848 9877 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc2444314f864d188196194f1cc75c46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc2444314f864d188196194f1cc75c46" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 40729 } } peers { permanent_uuid: "f9842102c66b4877b4b0a41200e58627" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 34651 } } peers { permanent_uuid: "ef9d5ff698a04c928be5f31171ed1b5f" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 41707 } }
I20260811 19:17:47.292169 9880 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc2444314f864d188196194f1cc75c46 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc2444314f864d188196194f1cc75c46. Latest consensus state: current_term: 1 leader_uuid: "fc2444314f864d188196194f1cc75c46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc2444314f864d188196194f1cc75c46" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 40729 } } peers { permanent_uuid: "f9842102c66b4877b4b0a41200e58627" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 34651 } } peers { permanent_uuid: "ef9d5ff698a04c928be5f31171ed1b5f" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 41707 } } }
I20260811 19:17:47.293042 9880 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc2444314f864d188196194f1cc75c46 [sys.catalog]: This master's current role is: LEADER
I20260811 19:17:47.295995 9893 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:17:47.314613 9893 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 19:17:47.338333 9813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:47.307239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45436 (local address 127.3.61.189:34651)
0811 19:17:47.307455 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:47.307461 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:47.308270 (+ 809us) server_negotiation.cc:407] Connection header received
0811 19:17:47.309469 (+ 1199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:47.309475 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:47.309565 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:47.309640 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:47.316802 (+ 7162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:47.317940 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:47.321588 (+ 3648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:47.322308 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:47.322412 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:47.329729 (+ 7317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:47.329737 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:47.329742 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:17:47.329790 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:47.333291 (+ 3501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:47.335320 (+ 2029us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:47.335327 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:47.335331 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:47.335529 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:47.336210 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:47.336216 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:47.336219 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:47.336674 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:47.336983 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:47.337636 (+ 653us) server_negotiation.cc:299] Negotiation successful
0811 19:17:47.337936 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 19:17:47.342536 9753 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ef9d5ff698a04c928be5f31171ed1b5f [term 1 FOLLOWER]: Refusing update from remote peer fc2444314f864d188196194f1cc75c46: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:17:47.343797 9877 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fc2444314f864d188196194f1cc75c46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef9d5ff698a04c928be5f31171ed1b5f" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 41707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 19:17:47.349128 9571 tablet.cc:2369] T 00000000000000000000000000000000 P f9842102c66b4877b4b0a41200e58627: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:17:47.352639 9603 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f9842102c66b4877b4b0a41200e58627 [term 1 FOLLOWER]: Refusing update from remote peer fc2444314f864d188196194f1cc75c46: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:17:47.354174 9877 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fc2444314f864d188196194f1cc75c46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9842102c66b4877b4b0a41200e58627" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 34651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:17:47.358461 9811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:47.298405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54082 (local address 127.3.61.190:40729)
0811 19:17:47.298591 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:47.298598 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:47.306278 (+ 7680us) server_negotiation.cc:407] Connection header received
0811 19:17:47.307277 (+ 999us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:47.307284 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:47.307387 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:47.307470 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:47.316999 (+ 9529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:47.318116 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:47.323268 (+ 5152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:47.324016 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:47.324119 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:47.334152 (+ 10033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:47.334159 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:47.334164 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:17:47.334213 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:47.342096 (+ 7883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:47.354719 (+ 12623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:47.354727 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:47.354732 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:47.354956 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:47.355501 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:47.355507 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:47.355511 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:47.355972 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:47.356323 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:47.357704 (+ 1381us) server_negotiation.cc:299] Negotiation successful
0811 19:17:47.358046 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 19:17:47.365432 9807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:47.299529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43808 (local address 127.3.61.188:41707)
0811 19:17:47.299729 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:47.299736 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:47.307146 (+ 7410us) server_negotiation.cc:407] Connection header received
0811 19:17:47.310738 (+ 3592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:47.310744 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:47.310887 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:47.310975 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:47.319768 (+ 8793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:47.320835 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:47.325170 (+ 4335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:47.325897 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:47.326010 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:47.335872 (+ 9862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:47.335886 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:47.335892 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:17:47.335942 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:47.340050 (+ 4108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:47.361203 (+ 21153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:47.361211 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:47.361215 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:47.361475 (+ 260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:47.362131 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:47.362138 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:47.362143 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:47.362617 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:47.362940 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:47.364252 (+ 1312us) server_negotiation.cc:299] Negotiation successful
0811 19:17:47.364823 (+ 571us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 19:17:47.378813 9816 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef9d5ff698a04c928be5f31171ed1b5f [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc2444314f864d188196194f1cc75c46. Latest consensus state: current_term: 1 leader_uuid: "fc2444314f864d188196194f1cc75c46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc2444314f864d188196194f1cc75c46" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 40729 } } peers { permanent_uuid: "f9842102c66b4877b4b0a41200e58627" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 34651 } } peers { permanent_uuid: "ef9d5ff698a04c928be5f31171ed1b5f" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 41707 } } }
I20260811 19:17:47.379106 9816 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef9d5ff698a04c928be5f31171ed1b5f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:17:47.382385 9911 mvcc.cc:205] Tried to move back new op lower bound from 7317405152600621056 to 7317405152399855616. Current Snapshot: MvccSnapshot[applied={T|T < 7317405152600621056}]
I20260811 19:17:47.395017 9880 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc2444314f864d188196194f1cc75c46 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fc2444314f864d188196194f1cc75c46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc2444314f864d188196194f1cc75c46" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 40729 } } peers { permanent_uuid: "f9842102c66b4877b4b0a41200e58627" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 34651 } } peers { permanent_uuid: "ef9d5ff698a04c928be5f31171ed1b5f" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 41707 } } }
I20260811 19:17:47.395372 9880 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc2444314f864d188196194f1cc75c46 [sys.catalog]: This master's current role is: LEADER
I20260811 19:17:47.402804 9842 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9842102c66b4877b4b0a41200e58627 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc2444314f864d188196194f1cc75c46. Latest consensus state: current_term: 1 leader_uuid: "fc2444314f864d188196194f1cc75c46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc2444314f864d188196194f1cc75c46" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 40729 } } peers { permanent_uuid: "f9842102c66b4877b4b0a41200e58627" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 34651 } } peers { permanent_uuid: "ef9d5ff698a04c928be5f31171ed1b5f" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 41707 } } }
I20260811 19:17:47.403156 9842 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9842102c66b4877b4b0a41200e58627 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:17:47.406462 9877 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc2444314f864d188196194f1cc75c46 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fc2444314f864d188196194f1cc75c46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc2444314f864d188196194f1cc75c46" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 40729 } } peers { permanent_uuid: "f9842102c66b4877b4b0a41200e58627" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 34651 } } peers { permanent_uuid: "ef9d5ff698a04c928be5f31171ed1b5f" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 41707 } } }
I20260811 19:17:47.406948 9877 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc2444314f864d188196194f1cc75c46 [sys.catalog]: This master's current role is: LEADER
I20260811 19:17:47.409549 9816 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef9d5ff698a04c928be5f31171ed1b5f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fc2444314f864d188196194f1cc75c46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc2444314f864d188196194f1cc75c46" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 40729 } } peers { permanent_uuid: "f9842102c66b4877b4b0a41200e58627" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 34651 } } peers { permanent_uuid: "ef9d5ff698a04c928be5f31171ed1b5f" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 41707 } } }
I20260811 19:17:47.409898 9816 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef9d5ff698a04c928be5f31171ed1b5f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:17:47.412372 9893 catalog_manager.cc:1347] Generated new cluster ID: 96aeec0b1ad540fd9c1dd6bac7844eba
I20260811 19:17:47.412490 9893 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:17:47.432387 9939 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 19:17:47.450318 9842 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9842102c66b4877b4b0a41200e58627 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fc2444314f864d188196194f1cc75c46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc2444314f864d188196194f1cc75c46" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 40729 } } peers { permanent_uuid: "f9842102c66b4877b4b0a41200e58627" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 34651 } } peers { permanent_uuid: "ef9d5ff698a04c928be5f31171ed1b5f" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 41707 } } }
I20260811 19:17:47.450623 9842 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9842102c66b4877b4b0a41200e58627 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:17:47.484012 9893 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:17:47.488024 9893 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:17:47.521420 9893 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fc2444314f864d188196194f1cc75c46: Generated new TSK 0
I20260811 19:17:47.522689 9893 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 19:17:47.683369 9811 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:17:47.660106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54098 (local address 127.3.61.190:40729)
0811 19:17:47.660374 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:47.660381 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:47.680924 (+ 20543us) server_negotiation.cc:407] Connection header received
0811 19:17:47.681544 (+ 620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:47.681550 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:47.681676 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:47.681772 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:47.683109 (+ 1337us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54098: BlockingRecv error: recv got EOF from 127.0.0.1:54098 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260811 19:17:47.687933 9807 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:17:47.666567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43818 (local address 127.3.61.188:41707)
0811 19:17:47.666796 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:47.666803 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:47.684421 (+ 17618us) server_negotiation.cc:407] Connection header received
0811 19:17:47.684627 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:47.684634 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:47.684726 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:47.684912 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:47.687640 (+ 2728us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43818: BlockingRecv error: recv got EOF from 127.0.0.1:43818 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260811 19:17:47.689620 9813 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:17:47.685815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45452 (local address 127.3.61.189:34651)
0811 19:17:47.686167 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:47.686173 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:47.687848 (+ 1675us) server_negotiation.cc:407] Connection header received
0811 19:17:47.688017 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:47.688022 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:47.688312 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:47.688524 (+ 212us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:47.689373 (+ 849us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45452: BlockingRecv error: recv got EOF from 127.0.0.1:45452 (error 108)
Metrics: {"server-negotiator.queue_time_us":215}
W20260811 19:17:47.755234 9813 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:17:47.730220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45466 (local address 127.3.61.189:34651)
0811 19:17:47.730448 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:47.730455 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:47.753411 (+ 22956us) server_negotiation.cc:407] Connection header received
0811 19:17:47.753713 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:47.753720 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:47.753892 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:47.754037 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:47.754959 (+ 922us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45466: BlockingRecv error: recv got EOF from 127.0.0.1:45466 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260811 19:17:47.776168 9813 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:17:47.771749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45476 (local address 127.3.61.189:34651)
0811 19:17:47.772019 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:47.772027 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:47.774373 (+ 2346us) server_negotiation.cc:407] Connection header received
0811 19:17:47.774614 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:47.774620 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:47.774729 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:47.774869 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:47.775909 (+ 1040us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45476: BlockingRecv error: recv got EOF from 127.0.0.1:45476 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
W20260811 19:17:47.792717 9813 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:17:47.787721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45484 (local address 127.3.61.189:34651)
0811 19:17:47.787955 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:47.787963 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:47.790738 (+ 2775us) server_negotiation.cc:407] Connection header received
0811 19:17:47.791027 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:47.791034 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:47.791148 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:47.791319 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:47.792450 (+ 1131us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45484: BlockingRecv error: recv got EOF from 127.0.0.1:45484 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
W20260811 19:17:47.804180 9813 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:17:47.801600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45486 (local address 127.3.61.189:34651)
0811 19:17:47.801939 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:47.801949 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:47.802550 (+ 601us) server_negotiation.cc:407] Connection header received
0811 19:17:47.802772 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:47.802778 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:47.802873 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:47.802984 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:47.803908 (+ 924us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45486: BlockingRecv error: recv got EOF from 127.0.0.1:45486 (error 108)
Metrics: {"server-negotiator.queue_time_us":168}
W20260811 19:17:47.826078 9813 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:17:47.820504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45500 (local address 127.3.61.189:34651)
0811 19:17:47.820709 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:47.820716 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:47.824315 (+ 3599us) server_negotiation.cc:407] Connection header received
0811 19:17:47.824564 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:47.824570 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:47.824668 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:47.824778 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:47.825810 (+ 1032us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45500: BlockingRecv error: recv got EOF from 127.0.0.1:45500 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260811 19:17:47.849329 9813 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:17:47.846635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45516 (local address 127.3.61.189:34651)
0811 19:17:47.846953 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:47.846960 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:47.847393 (+ 433us) server_negotiation.cc:407] Connection header received
0811 19:17:47.847673 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:47.847679 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:47.847774 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:47.847888 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:47.849017 (+ 1129us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45516: BlockingRecv error: recv got EOF from 127.0.0.1:45516 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
W20260811 19:17:47.887741 9813 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:17:47.885506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45520 (local address 127.3.61.189:34651)
0811 19:17:47.885775 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:47.885782 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:47.886176 (+ 394us) server_negotiation.cc:407] Connection header received
0811 19:17:47.886332 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:47.886339 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:47.886500 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:47.886630 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:47.887464 (+ 834us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45520: BlockingRecv error: recv got EOF from 127.0.0.1:45520 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260811 19:17:47.958735 9813 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:17:47.956450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45526 (local address 127.3.61.189:34651)
0811 19:17:47.956699 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:47.956706 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:47.957240 (+ 534us) server_negotiation.cc:407] Connection header received
0811 19:17:47.957429 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:47.957435 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:47.957527 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:47.957638 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:47.958508 (+ 870us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45526: BlockingRecv error: recv got EOF from 127.0.0.1:45526 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
W20260811 19:17:48.098786 9813 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:17:48.096474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45542 (local address 127.3.61.189:34651)
0811 19:17:48.096785 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:48.096793 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:48.097111 (+ 318us) server_negotiation.cc:407] Connection header received
0811 19:17:48.097357 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:48.097362 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:48.097470 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:48.097570 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:48.098544 (+ 974us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45542: BlockingRecv error: recv got EOF from 127.0.0.1:45542 (error 108)
Metrics: {"server-negotiator.queue_time_us":119}
I20260811 19:17:48.127271 9835 catalog_manager.cc:1259] Loaded cluster ID: 96aeec0b1ad540fd9c1dd6bac7844eba
I20260811 19:17:48.127388 9835 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ef9d5ff698a04c928be5f31171ed1b5f: loading cluster ID for follower catalog manager: success
I20260811 19:17:48.130028 9835 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ef9d5ff698a04c928be5f31171ed1b5f: acquiring CA information for follower catalog manager: success
I20260811 19:17:48.132063 9835 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ef9d5ff698a04c928be5f31171ed1b5f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 19:17:48.188112 9871 catalog_manager.cc:1259] Loaded cluster ID: 96aeec0b1ad540fd9c1dd6bac7844eba
I20260811 19:17:48.188235 9871 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f9842102c66b4877b4b0a41200e58627: loading cluster ID for follower catalog manager: success
I20260811 19:17:48.191542 9871 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f9842102c66b4877b4b0a41200e58627: acquiring CA information for follower catalog manager: success
I20260811 19:17:48.194202 9871 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f9842102c66b4877b4b0a41200e58627: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 19:17:48.365483 9813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:48.359316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45550 (local address 127.3.61.189:34651)
0811 19:17:48.359532 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:48.359537 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:48.359775 (+ 238us) server_negotiation.cc:407] Connection header received
0811 19:17:48.359912 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:48.359916 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:48.360115 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:48.360233 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:17:48.360723 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:48.361478 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:48.362943 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:48.363447 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:48.363561 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:48.364174 (+ 613us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:17:48.364826 (+ 652us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:17:48.364948 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:48.365117 (+ 169us) server_negotiation.cc:299] Negotiation successful
0811 19:17:48.365195 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260811 19:17:48.392498 10070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:48.384310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43822 (local address 127.3.61.188:41707)
0811 19:17:48.385406 (+ 1096us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:48.385411 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:48.385458 (+ 47us) server_negotiation.cc:407] Connection header received
0811 19:17:48.385567 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:48.385572 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:48.385841 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:48.386012 (+ 171us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:17:48.386731 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:48.387672 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:48.389396 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:48.390000 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:48.390147 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:48.390989 (+ 842us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:17:48.391620 (+ 631us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:17:48.391765 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:48.392017 (+ 252us) server_negotiation.cc:299] Negotiation successful
0811 19:17:48.392119 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":987,"thread_start_us":885,"threads_started":1}
I20260811 19:17:48.411209 10070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:48.403176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43828 (local address 127.3.61.188:41707)
0811 19:17:48.403422 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:48.403429 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:48.403610 (+ 181us) server_negotiation.cc:407] Connection header received
0811 19:17:48.403883 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:48.403890 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:48.404020 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:48.404137 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:17:48.404898 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:48.406053 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:48.408660 (+ 2607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:48.409475 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:48.409643 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:48.410310 (+ 667us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:17:48.410477 (+ 167us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:17:48.410581 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:48.410743 (+ 162us) server_negotiation.cc:299] Negotiation successful
0811 19:17:48.410844 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 19:17:48.412596 10078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:48.402789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54114 (local address 127.3.61.190:40729)
0811 19:17:48.404230 (+ 1441us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:48.404237 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:48.404885 (+ 648us) server_negotiation.cc:407] Connection header received
0811 19:17:48.405258 (+ 373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:48.405265 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:48.405560 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:48.405803 (+ 243us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:17:48.406490 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:48.407441 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:48.409929 (+ 2488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:48.410499 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:48.410623 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:48.411380 (+ 757us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:17:48.411899 (+ 519us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:17:48.412012 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:48.412162 (+ 150us) server_negotiation.cc:299] Negotiation successful
0811 19:17:48.412275 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1315,"thread_start_us":1212,"threads_started":1}
I20260811 19:17:48.412597 9813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:48.405067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45552 (local address 127.3.61.189:34651)
0811 19:17:48.405296 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:48.405303 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:48.405874 (+ 571us) server_negotiation.cc:407] Connection header received
0811 19:17:48.406172 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:48.406178 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:48.406279 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:48.406379 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:17:48.407124 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:48.408452 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:48.410488 (+ 2036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:48.411205 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:48.411344 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:48.411794 (+ 450us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:17:48.411924 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:17:48.412012 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:48.412209 (+ 197us) server_negotiation.cc:299] Negotiation successful
0811 19:17:48.412300 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 19:17:48.418749 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 9369
I20260811 19:17:48.419869 9512 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:17:48.534314 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 9369
I20260811 19:17:48.563413 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 9530
I20260811 19:17:48.564730 9639 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:17:48.679992 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 9530
I20260811 19:17:48.706017 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 9668
I20260811 19:17:48.707378 9790 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:17:48.823539 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 9668
2026-08-11T19:17:48Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5443 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 19:17:48 dist-test-slave-t7g5 krb5kdc[10098](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 19:17:48 dist-test-slave-t7g5 krb5kdc[10098](info): set up 2 sockets
Aug 11 19:17:48 dist-test-slave-t7g5 krb5kdc[10098](info): commencing operation
krb5kdc: starting...
W20260811 19:17:50.205454 3318 mini_kdc.cc:121] Time spent starting KDC: real 1.327s user 0.003s sys 0.064s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/krb5kdc/test-user.keytab.
Aug 11 19:17:50 dist-test-slave-t7g5 krb5kdc[10098](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475870, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T19:17:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T19:17:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.61.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab.
Entry for principal kudu/127.3.61.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab.
I20260811 19:17:50.639210 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:44989
--webserver_interface=127.3.61.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:42007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.61.190:44989,127.3.61.189:45797,127.3.61.188:39275
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 19:17:51.080586 10123 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 19:17:51.081146 10123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:17:51.081282 10123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:17:51.081427 10123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:17:51.090719 10123 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:17:51.090881 10123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:17:51.090957 10123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:17:51.091035 10123 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:17:51.091141 10123 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:17:51.105072 10123 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:42007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.61.190:44989,127.3.61.189:45797,127.3.61.188:39275
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.190.keytab
--principal=kudu/127.3.61.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.190:44989
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.61.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:17:51.106714 10123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:17:51.108249 10123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:17:51.122481 10135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:17:51.122383 10141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:17:51.122336 10133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:17:51.128060 10123 server_base.cc:1034] running on GCE node
I20260811 19:17:51.129150 10123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:17:51.131074 10123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:17:51.132596 10123 hybrid_clock.cc:648] HybridClock initialized: now 1786475871132475 us; error 77 us; skew 500 ppm
I20260811 19:17:51.133036 10123 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:17:51 dist-test-slave-t7g5 krb5kdc[10098](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475871, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:17:51.142133 10123 init.cc:377] Logged in from keytab as kudu/127.3.61.190@KRBTEST.COM (short username kudu)
I20260811 19:17:51.146325 10123 webserver.cc:460] Webserver started at http://127.3.61.190:45275/ using document root <none> and password file <none>
I20260811 19:17:51.147253 10123 fs_manager.cc:359] Metadata directory not provided
I20260811 19:17:51.147401 10123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:17:51.147780 10123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:17:51.151333 10123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/minicluster-data/master-0/data/instance:
uuid: "38f4bd77341547e5bbd03a3d521b6b6d"
format_stamp: "Formatted at 2026-08-11 19:17:51 on dist-test-slave-t7g5"
server_key: "67e061844708001268d00f614c5671e7"
server_key_iv: "db7b0bc4c6cfe3120aac4c4aff5efb46"
server_key_version: "encryptionkey@0"
I20260811 19:17:51.152231 10123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/minicluster-data/master-0/wal/instance:
uuid: "38f4bd77341547e5bbd03a3d521b6b6d"
format_stamp: "Formatted at 2026-08-11 19:17:51 on dist-test-slave-t7g5"
server_key: "67e061844708001268d00f614c5671e7"
server_key_iv: "db7b0bc4c6cfe3120aac4c4aff5efb46"
server_key_version: "encryptionkey@0"
I20260811 19:17:51.160811 10123 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.009s sys 0.000s
I20260811 19:17:51.166481 10153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:17:51.168399 10123 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260811 19:17:51.168660 10123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/minicluster-data/master-0/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/minicluster-data/master-0/wal
uuid: "38f4bd77341547e5bbd03a3d521b6b6d"
format_stamp: "Formatted at 2026-08-11 19:17:51 on dist-test-slave-t7g5"
server_key: "67e061844708001268d00f614c5671e7"
server_key_iv: "db7b0bc4c6cfe3120aac4c4aff5efb46"
server_key_version: "encryptionkey@0"
I20260811 19:17:51.168978 10123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:17:51.189116 10123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:17:51.196084 10123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:17:51.196558 10123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:17:51.221349 10235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.190:44989 every 8 connection(s)
I20260811 19:17:51.221340 10123 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.190:44989
I20260811 19:17:51.223148 10123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/minicluster-data/master-0/data/info.pb
I20260811 19:17:51.227706 10236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:17:51.230726 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 10123
I20260811 19:17:51.230945 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/minicluster-data/master-0/wal/instance
I20260811 19:17:51.231398 3318 external_mini_cluster.cc:1428] Setting key 4dca4bae6d222a3842fa254b667c5bcd
I20260811 19:17:51.235579 10236 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.61.190" port: 44989 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 19:17:51 dist-test-slave-t7g5 krb5kdc[10098](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475871, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.190@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:17:51.267906 10241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:51.239810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:44989 (local address 127.0.0.1:51852)
0811 19:17:51.241147 (+ 1337us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:51.241233 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0811 19:17:51.244118 (+ 2885us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:17:51.245565 (+ 1447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:17:51.245573 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:17:51.246471 (+ 898us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:17:51.246982 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:17:51.247009 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:17:51.249583 (+ 2574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:17:51.249589 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:17:51.250962 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:17:51.250991 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:17:51.252785 (+ 1794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:17:51.252791 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:17:51.252957 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:51.253874 (+ 917us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:17:51.253910 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:17:51.257316 (+ 3406us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:17:51.263041 (+ 5725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:17:51.263057 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:17:51.263069 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:17:51.263596 (+ 527us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:17:51.264187 (+ 591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:17:51.264191 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:17:51.264194 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:17:51.264420 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:17:51.265615 (+ 1195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:17:51.265622 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:17:51.265880 (+ 258us) client_negotiation.cc:769] Sending connection context
0811 19:17:51.266163 (+ 283us) client_negotiation.cc:240] Negotiation successful
0811 19:17:51.266675 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":917,"spinlock_wait_cycles":18816,"thread_start_us":590,"threads_started":1}
I20260811 19:17:51.267961 10242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:51.241822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51852 (local address 127.3.61.190:44989)
0811 19:17:51.242735 (+ 913us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:51.242743 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:51.242813 (+ 70us) server_negotiation.cc:407] Connection header received
0811 19:17:51.244777 (+ 1964us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:51.244787 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:51.245211 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:51.245589 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:51.247386 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:51.249212 (+ 1826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:51.251218 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:51.252353 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:51.252784 (+ 431us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:51.257852 (+ 5068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:51.257869 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:51.257891 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:17:51.257961 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:51.262812 (+ 4851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:51.263798 (+ 986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:51.263808 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:51.263811 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:51.264020 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:51.264565 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:51.264570 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:51.264573 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:51.265428 (+ 855us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:51.265701 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:51.266370 (+ 669us) server_negotiation.cc:299] Negotiation successful
0811 19:17:51.266901 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":175,"server-negotiator.queue_time_us":675,"thread_start_us":462,"threads_started":1}
I20260811 19:17:51.272626 10236 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.61.189" port: 45797 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.3.61.189@KRBTEST.COM; defaulting to no policy
I20260811 19:17:51.274915 10241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:51.273956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.189:45797 (local address 127.0.0.1:58094)
0811 19:17:51.274266 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:51.274498 (+ 232us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.189:45797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
W20260811 19:17:51.275388 10156 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.61.189:45797: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.189@KRBTEST.COM" created.
I20260811 19:17:51.277544 10241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:51.276948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.189:45797 (local address 127.0.0.1:58106)
0811 19:17:51.277196 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:51.277310 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.189:45797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
W20260811 19:17:51.277905 10236 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.61.189:45797: Network error: Client connection negotiation failed: client connection to 127.3.61.189:45797: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.189.keytab.
Entry for principal kudu/127.3.61.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.189.keytab.
I20260811 19:17:51.327292 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.189:45797
--webserver_interface=127.3.61.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.189.keytab
--principal=kudu/127.3.61.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:42007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.61.190:44989,127.3.61.189:45797,127.3.61.188:39275
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:17:51.327500 10236 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.61.189" port: 45797 } attempt: 1
I20260811 19:17:51.329128 10241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:51.328493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.189:45797 (local address 127.0.0.1:58120)
0811 19:17:51.328802 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:51.328964 (+ 162us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.189:45797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
I20260811 19:17:51.330641 10241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:51.330001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.189:45797 (local address 127.0.0.1:58128)
0811 19:17:51.330285 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:51.330386 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.189:45797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126}
W20260811 19:17:51.330950 10236 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.61.189:45797: Network error: Client connection negotiation failed: client connection to 127.3.61.189:45797: connect: Connection refused (error 111)
I20260811 19:17:51.399319 10236 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.61.189" port: 45797 } attempt: 2
I20260811 19:17:51.401075 10241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:51.400402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.189:45797 (local address 127.0.0.1:58134)
0811 19:17:51.400738 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:51.400864 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.189:45797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161}
I20260811 19:17:51.402880 10241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:51.402251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.189:45797 (local address 127.0.0.1:58146)
0811 19:17:51.402528 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:51.402683 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.189:45797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
W20260811 19:17:51.403223 10236 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.61.189:45797: Network error: Client connection negotiation failed: client connection to 127.3.61.189:45797: connect: Connection refused (error 111)
I20260811 19:17:51.494625 10236 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.61.189" port: 45797 } attempt: 3
I20260811 19:17:51.495970 10241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:51.495473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.189:45797 (local address 127.0.0.1:58148)
0811 19:17:51.495697 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:51.495795 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.189:45797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260811 19:17:51.497283 10241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:51.496830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.189:45797 (local address 127.0.0.1:58160)
0811 19:17:51.497020 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:51.497106 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.189:45797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260811 19:17:51.497596 10236 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.61.189:45797: Network error: Client connection negotiation failed: client connection to 127.3.61.189:45797: connect: Connection refused (error 111)
I20260811 19:17:51.641065 10236 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.61.189" port: 45797 } attempt: 4
I20260811 19:17:51.642988 10241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:51.642174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.189:45797 (local address 127.0.0.1:58168)
0811 19:17:51.642685 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:51.642808 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.189:45797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":290}
I20260811 19:17:51.644663 10241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:51.643868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.189:45797 (local address 127.0.0.1:58180)
0811 19:17:51.644244 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:51.644442 (+ 198us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.189:45797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145}
W20260811 19:17:51.645015 10236 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.61.189:45797: Network error: Client connection negotiation failed: client connection to 127.3.61.189:45797: connect: Connection refused (error 111)
W20260811 19:17:51.755868 10250 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 19:17:51.756367 10250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:17:51.756482 10250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:17:51.756626 10250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:17:51.765617 10250 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:17:51.765836 10250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:17:51.765918 10250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:17:51.766001 10250 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:17:51.766081 10250 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:17:51.778272 10250 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:42007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.3.61.190:44989,127.3.61.189:45797,127.3.61.188:39275
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.189.keytab
--principal=kudu/127.3.61.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.189:45797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.3.61.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:17:51.779832 10250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:17:51.781476 10250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:17:51.794718 10261 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:17:51.795027 10265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:17:51.795050 10259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:17:51.796085 10250 server_base.cc:1034] running on GCE node
I20260811 19:17:51.797071 10250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:17:51.798936 10250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:17:51.800276 10250 hybrid_clock.cc:648] HybridClock initialized: now 1786475871800165 us; error 79 us; skew 500 ppm
I20260811 19:17:51.800873 10250 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:17:51 dist-test-slave-t7g5 krb5kdc[10098](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475871, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:17:51.805256 10250 init.cc:377] Logged in from keytab as kudu/127.3.61.189@KRBTEST.COM (short username kudu)
I20260811 19:17:51.807490 10250 webserver.cc:460] Webserver started at http://127.3.61.189:33863/ using document root <none> and password file <none>
I20260811 19:17:51.808672 10250 fs_manager.cc:359] Metadata directory not provided
I20260811 19:17:51.809053 10250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:17:51.809630 10250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:17:51.814453 10250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/minicluster-data/master-1/data/instance:
uuid: "bdfac25e5f034b68b2f140192370d2b4"
format_stamp: "Formatted at 2026-08-11 19:17:51 on dist-test-slave-t7g5"
server_key: "b3ec20a0c59435ab625bac9c8a00fcbd"
server_key_iv: "8a11ae65e5583fe5dfbca49db2a369df"
server_key_version: "encryptionkey@0"
I20260811 19:17:51.815685 10250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/minicluster-data/master-1/wal/instance:
uuid: "bdfac25e5f034b68b2f140192370d2b4"
format_stamp: "Formatted at 2026-08-11 19:17:51 on dist-test-slave-t7g5"
server_key: "b3ec20a0c59435ab625bac9c8a00fcbd"
server_key_iv: "8a11ae65e5583fe5dfbca49db2a369df"
server_key_version: "encryptionkey@0"
I20260811 19:17:51.823611 10250 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.006s
I20260811 19:17:51.828980 10282 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:17:51.830883 10250 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.002s
I20260811 19:17:51.831110 10250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/minicluster-data/master-1/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/minicluster-data/master-1/wal
uuid: "bdfac25e5f034b68b2f140192370d2b4"
format_stamp: "Formatted at 2026-08-11 19:17:51 on dist-test-slave-t7g5"
server_key: "b3ec20a0c59435ab625bac9c8a00fcbd"
server_key_iv: "8a11ae65e5583fe5dfbca49db2a369df"
server_key_version: "encryptionkey@0"
I20260811 19:17:51.831400 10250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:17:51.848188 10250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:17:51.854290 10250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:17:51.854610 10250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:17:51.878669 10250 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.189:45797
I20260811 19:17:51.878964 10382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.189:45797 every 8 connection(s)
I20260811 19:17:51.880538 10250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/minicluster-data/master-1/data/info.pb
I20260811 19:17:51.885414 10383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:17:51.890407 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 10250
I20260811 19:17:51.890738 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/minicluster-data/master-1/wal/instance
I20260811 19:17:51.891436 3318 external_mini_cluster.cc:1428] Setting key 99c60a8aefbe1f81487186b6a02ad697
I20260811 19:17:51.896517 10383 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.61.190" port: 44989 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 19:17:51 dist-test-slave-t7g5 krb5kdc[10098](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475871, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.189@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:17:51.926512 10385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:51.900423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51868 (local address 127.3.61.190:44989)
0811 19:17:51.901291 (+ 868us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:51.901298 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:51.902610 (+ 1312us) server_negotiation.cc:407] Connection header received
0811 19:17:51.905714 (+ 3104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:51.905722 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:51.905835 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:51.905956 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:51.908504 (+ 2548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:51.909271 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:51.911658 (+ 2387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:51.912308 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:51.912488 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:51.918196 (+ 5708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:51.918204 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:51.918209 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:17:51.918275 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:51.921711 (+ 3436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:51.923077 (+ 1366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:51.923084 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:51.923087 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:51.923319 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:51.924154 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:51.924161 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:51.924164 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:51.924625 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:51.924792 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:51.925966 (+ 1174us) server_negotiation.cc:299] Negotiation successful
0811 19:17:51.926203 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":707,"thread_start_us":406,"threads_started":1}
I20260811 19:17:51.928613 10386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:51.901389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:44989 (local address 127.0.0.1:51868)
0811 19:17:51.902371 (+ 982us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:51.902440 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0811 19:17:51.905177 (+ 2737us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:17:51.906322 (+ 1145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:17:51.906335 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:17:51.907208 (+ 873us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:17:51.908121 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:17:51.908154 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:17:51.909480 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:17:51.909486 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:17:51.911416 (+ 1930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:17:51.911437 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:17:51.912803 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:17:51.912812 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:17:51.913182 (+ 370us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:51.914231 (+ 1049us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:17:51.914271 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:17:51.917711 (+ 3440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:17:51.921970 (+ 4259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:17:51.921984 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:17:51.922007 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:17:51.922797 (+ 790us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:17:51.923556 (+ 759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:17:51.923561 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:17:51.923565 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:17:51.923918 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:17:51.924819 (+ 901us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:17:51.924831 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:17:51.925403 (+ 572us) client_negotiation.cc:769] Sending connection context
0811 19:17:51.925784 (+ 381us) client_negotiation.cc:240] Negotiation successful
0811 19:17:51.926599 (+ 815us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":645,"thread_start_us":391,"threads_started":1}
I20260811 19:17:51.931218 10383 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.61.189" port: 45797 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.3.61.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.61.188@KRBTEST.COM" created.
Aug 11 19:17:51 dist-test-slave-t7g5 krb5kdc[10098](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475871, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.189@KRBTEST.COM for kudu/127.3.61.189@KRBTEST.COM
I20260811 19:17:51.944301 10236 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.61.189" port: 45797 } attempt: 5
I20260811 19:17:51.953641 10386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:51.932233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.189:45797 (local address 127.0.0.1:58194)
0811 19:17:51.932450 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:51.932472 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 19:17:51.932580 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:17:51.934076 (+ 1496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:17:51.934081 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:17:51.934907 (+ 826us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:17:51.935190 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:17:51.935207 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:17:51.937169 (+ 1962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:17:51.937174 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:17:51.938377 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:17:51.938395 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:17:51.939748 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:17:51.939756 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:17:51.939933 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:51.940657 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:17:51.940683 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:17:51.943889 (+ 3206us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:17:51.949996 (+ 6107us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:17:51.950001 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:17:51.950005 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:17:51.950576 (+ 571us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:17:51.951038 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:17:51.951043 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:17:51.951045 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:17:51.951244 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:17:51.952223 (+ 979us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:17:51.952227 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:17:51.952349 (+ 122us) client_negotiation.cc:769] Sending connection context
0811 19:17:51.952475 (+ 126us) client_negotiation.cc:240] Negotiation successful
0811 19:17:51.953281 (+ 806us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64,"mutex_wait_us":135}
I20260811 19:17:51.953686 10388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:51.932442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58194 (local address 127.3.61.189:45797)
0811 19:17:51.933326 (+ 884us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:51.933336 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:51.933378 (+ 42us) server_negotiation.cc:407] Connection header received
0811 19:17:51.933470 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:51.933479 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:51.933857 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:51.934094 (+ 237us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:51.935629 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:51.936970 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:51.938769 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:51.939436 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:51.939566 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:51.944236 (+ 4670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:51.944261 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:51.944286 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:17:51.944334 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:51.949862 (+ 5528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:51.950739 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:51.950749 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:51.950753 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:51.950927 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:51.951349 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:51.951354 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:51.951357 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:51.951932 (+ 575us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:51.952166 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:51.952687 (+ 521us) server_negotiation.cc:299] Negotiation successful
0811 19:17:51.953118 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":96,"server-negotiator.queue_time_us":665,"thread_start_us":456,"threads_started":1}
Aug 11 19:17:51 dist-test-slave-t7g5 krb5kdc[10098](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475871, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.190@KRBTEST.COM for kudu/127.3.61.189@KRBTEST.COM
I20260811 19:17:51.958164 10383 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.61.188" port: 39275 } has no permanent_uuid. Determining permanent_uuid...
I20260811 19:17:51.959789 10386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:51.959137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:39275 (local address 127.0.0.1:57604)
0811 19:17:51.959381 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:51.959553 (+ 172us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.188:39275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
W20260811 19:17:51.960129 10288 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.61.188:39275: connect: Connection refused (error 111)
I20260811 19:17:51.962378 10386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:51.961897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:39275 (local address 127.0.0.1:57618)
0811 19:17:51.962083 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:51.962208 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.188:39275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260811 19:17:51.962935 10383 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.61.188:39275: Network error: Client connection negotiation failed: client connection to 127.3.61.188:39275: connect: Connection refused (error 111)
I20260811 19:17:51.964356 10241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:51.945214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.189:45797 (local address 127.0.0.1:58208)
0811 19:17:51.945435 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:51.945458 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 19:17:51.945588 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:17:51.947284 (+ 1696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:17:51.947288 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:17:51.947773 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:17:51.948021 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:17:51.948031 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:17:51.949827 (+ 1796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:17:51.949832 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:17:51.951027 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:17:51.951045 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:17:51.951984 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:17:51.951990 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:17:51.952241 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:51.952981 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:17:51.953004 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:17:51.955852 (+ 2848us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:17:51.961095 (+ 5243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:17:51.961104 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:17:51.961108 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:17:51.961953 (+ 845us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:17:51.962393 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:17:51.962397 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:17:51.962400 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:17:51.962586 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:17:51.963280 (+ 694us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:17:51.963285 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:17:51.963485 (+ 200us) client_negotiation.cc:769] Sending connection context
0811 19:17:51.963649 (+ 164us) client_negotiation.cc:240] Negotiation successful
0811 19:17:51.963998 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 19:17:51.964426 10391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:51.945770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58208 (local address 127.3.61.189:45797)
0811 19:17:51.946687 (+ 917us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:51.946694 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:51.946728 (+ 34us) server_negotiation.cc:407] Connection header received
0811 19:17:51.946812 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:51.946925 (+ 113us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:51.947053 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:51.947221 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:51.948264 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:51.949626 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:51.951163 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:51.951810 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:51.951943 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:51.956322 (+ 4379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:51.956339 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:51.956346 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:17:51.956511 (+ 165us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:51.960782 (+ 4271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:51.962103 (+ 1321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:51.962107 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:51.962110 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:51.962257 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:51.962728 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:51.962733 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:51.962736 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:51.963128 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:51.963276 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:51.963770 (+ 494us) server_negotiation.cc:299] Negotiation successful
0811 19:17:51.964006 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":727,"thread_start_us":382,"threads_started":1}
I20260811 19:17:51.965607 10236 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.61.188" port: 39275 } has no permanent_uuid. Determining permanent_uuid...
I20260811 19:17:51.967257 10241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:51.966487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:39275 (local address 127.0.0.1:57630)
0811 19:17:51.966801 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:51.966949 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.188:39275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138}
I20260811 19:17:51.968987 10241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:51.968324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:39275 (local address 127.0.0.1:57642)
0811 19:17:51.968502 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:51.968780 (+ 278us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.188:39275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260811 19:17:51.969331 10236 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.61.188:39275: Network error: Client connection negotiation failed: client connection to 127.3.61.188:39275: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.61.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.188.keytab.
Entry for principal kudu/127.3.61.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.188.keytab.
W20260811 19:17:51.993187 3318 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.3.61.188: real 0.101s user 0.001s sys 0.043s
I20260811 19:17:51.994069 3318 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
/tmp/dist-test-taskwEqXVd/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.188:39275
--webserver_interface=127.3.61.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.188.keytab
--principal=kudu/127.3.61.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.61.148:42007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.61.190:44989,127.3.61.189:45797,127.3.61.188:39275
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 19:17:52.012544 10383 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.61.188" port: 39275 } attempt: 1
I20260811 19:17:52.014317 10386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:52.013632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:39275 (local address 127.0.0.1:57658)
0811 19:17:52.013916 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:52.014085 (+ 169us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.188:39275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
I20260811 19:17:52.015977 10386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:52.015413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:39275 (local address 127.0.0.1:57660)
0811 19:17:52.015665 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:52.015791 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.188:39275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
W20260811 19:17:52.016454 10383 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.61.188:39275: Network error: Client connection negotiation failed: client connection to 127.3.61.188:39275: connect: Connection refused (error 111)
I20260811 19:17:52.020653 10236 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.61.188" port: 39275 } attempt: 1
I20260811 19:17:52.022202 10241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:52.021507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:39275 (local address 127.0.0.1:57666)
0811 19:17:52.021802 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:52.021995 (+ 193us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.188:39275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260811 19:17:52.023636 10241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:52.023197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:39275 (local address 127.0.0.1:57668)
0811 19:17:52.023417 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:52.023502 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.188:39275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
W20260811 19:17:52.024029 10236 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.61.188:39275: Network error: Client connection negotiation failed: client connection to 127.3.61.188:39275: connect: Connection refused (error 111)
I20260811 19:17:52.084795 10383 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.61.188" port: 39275 } attempt: 2
I20260811 19:17:52.086072 10386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:52.085559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:39275 (local address 127.0.0.1:57684)
0811 19:17:52.085799 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:52.085893 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.188:39275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260811 19:17:52.087272 10386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:52.086914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:39275 (local address 127.0.0.1:57698)
0811 19:17:52.087057 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:52.087135 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.188:39275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 19:17:52.087527 10383 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.61.188:39275: Network error: Client connection negotiation failed: client connection to 127.3.61.188:39275: connect: Connection refused (error 111)
I20260811 19:17:52.092384 10236 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.61.188" port: 39275 } attempt: 2
I20260811 19:17:52.093621 10241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:52.093107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:39275 (local address 127.0.0.1:57704)
0811 19:17:52.093331 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:52.093445 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.188:39275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260811 19:17:52.095422 10241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:52.094787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:39275 (local address 127.0.0.1:57720)
0811 19:17:52.095088 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:52.095213 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.188:39275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
W20260811 19:17:52.095788 10236 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.61.188:39275: Network error: Client connection negotiation failed: client connection to 127.3.61.188:39275: connect: Connection refused (error 111)
I20260811 19:17:52.178826 10383 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.61.188" port: 39275 } attempt: 3
I20260811 19:17:52.180562 10386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:52.179796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:39275 (local address 127.0.0.1:57734)
0811 19:17:52.180163 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:52.180331 (+ 168us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.188:39275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260811 19:17:52.182156 10386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:52.181465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:39275 (local address 127.0.0.1:57740)
0811 19:17:52.181817 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:52.181940 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.188:39275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239}
W20260811 19:17:52.182487 10383 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.61.188:39275: Network error: Client connection negotiation failed: client connection to 127.3.61.188:39275: connect: Connection refused (error 111)
I20260811 19:17:52.202075 10236 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.61.188" port: 39275 } attempt: 3
I20260811 19:17:52.203744 10241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:52.203126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:39275 (local address 127.0.0.1:57742)
0811 19:17:52.203413 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:52.203536 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.188:39275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260811 19:17:52.204907 10241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:52.204484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:39275 (local address 127.0.0.1:57750)
0811 19:17:52.204707 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:52.204785 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.188:39275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
W20260811 19:17:52.205149 10236 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.61.188:39275: Network error: Client connection negotiation failed: client connection to 127.3.61.188:39275: connect: Connection refused (error 111)
I20260811 19:17:52.325848 10383 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.61.188" port: 39275 } attempt: 4
I20260811 19:17:52.327679 10386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:52.326980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:39275 (local address 127.0.0.1:57766)
0811 19:17:52.327335 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:52.327448 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.188:39275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166}
I20260811 19:17:52.329038 10386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:52.328608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:39275 (local address 127.0.0.1:57768)
0811 19:17:52.328811 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:52.328902 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.188:39275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
W20260811 19:17:52.329366 10383 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.61.188:39275: Network error: Client connection negotiation failed: client connection to 127.3.61.188:39275: connect: Connection refused (error 111)
I20260811 19:17:52.382431 10236 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.61.188" port: 39275 } attempt: 4
I20260811 19:17:52.383669 10241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:52.383203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:39275 (local address 127.0.0.1:57776)
0811 19:17:52.383422 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:52.383508 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.188:39275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260811 19:17:52.384981 10241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:52.384577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:39275 (local address 127.0.0.1:57780)
0811 19:17:52.384760 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:52.384837 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.61.188:39275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
W20260811 19:17:52.385200 10236 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.61.188:39275: Network error: Client connection negotiation failed: client connection to 127.3.61.188:39275: connect: Connection refused (error 111)
W20260811 19:17:52.426726 10395 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 19:17:52.427263 10395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 19:17:52.427368 10395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 19:17:52.427490 10395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 19:17:52.435364 10395 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 19:17:52.435544 10395 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 19:17:52.435667 10395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 19:17:52.435737 10395 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 19:17:52.435847 10395 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 19:17:52.446784 10395 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.61.148:42007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.3.61.190:44989,127.3.61.189:45797,127.3.61.188:39275
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/krb5kdc/kudu_127.3.61.188.keytab
--principal=kudu/127.3.61.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.61.188:39275
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.3.61.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 18:08:14 UTC on ccycloud.quasar-lktbbr.root.comops.site
build id 4581130
ASAN enabled
I20260811 19:17:52.448266 10395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 19:17:52.449906 10395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 19:17:52.464200 10413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:17:52.464191 10405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 19:17:52.464610 10406 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 19:17:52.465241 10395 server_base.cc:1034] running on GCE node
I20260811 19:17:52.466361 10395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 19:17:52.468245 10395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 19:17:52.471865 10395 hybrid_clock.cc:648] HybridClock initialized: now 1786475872471717 us; error 92 us; skew 500 ppm
I20260811 19:17:52.472472 10395 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 19:17:52 dist-test-slave-t7g5 krb5kdc[10098](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475872, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 19:17:52.477970 10395 init.cc:377] Logged in from keytab as kudu/127.3.61.188@KRBTEST.COM (short username kudu)
I20260811 19:17:52.479954 10395 webserver.cc:460] Webserver started at http://127.3.61.188:41295/ using document root <none> and password file <none>
I20260811 19:17:52.480863 10395 fs_manager.cc:359] Metadata directory not provided
I20260811 19:17:52.481007 10395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 19:17:52.481397 10395 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 19:17:52.485144 10395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/minicluster-data/master-2/data/instance:
uuid: "3c1725bffbdb452fa4ce066f15ac5c38"
format_stamp: "Formatted at 2026-08-11 19:17:52 on dist-test-slave-t7g5"
server_key: "3c773d921e9d176c1308746bfd5802cc"
server_key_iv: "488b529458362d25ddf775256ed191e0"
server_key_version: "encryptionkey@0"
I20260811 19:17:52.486178 10395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/minicluster-data/master-2/wal/instance:
uuid: "3c1725bffbdb452fa4ce066f15ac5c38"
format_stamp: "Formatted at 2026-08-11 19:17:52 on dist-test-slave-t7g5"
server_key: "3c773d921e9d176c1308746bfd5802cc"
server_key_iv: "488b529458362d25ddf775256ed191e0"
server_key_version: "encryptionkey@0"
I20260811 19:17:52.492506 10395 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.001s
I20260811 19:17:52.497594 10426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 19:17:52.499881 10395 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260811 19:17:52.500180 10395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/minicluster-data/master-2/data,/tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/minicluster-data/master-2/wal
uuid: "3c1725bffbdb452fa4ce066f15ac5c38"
format_stamp: "Formatted at 2026-08-11 19:17:52 on dist-test-slave-t7g5"
server_key: "3c773d921e9d176c1308746bfd5802cc"
server_key_iv: "488b529458362d25ddf775256ed191e0"
server_key_version: "encryptionkey@0"
I20260811 19:17:52.500456 10395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 19:17:52.520077 10395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 19:17:52.525074 10395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 19:17:52.525403 10395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 19:17:52.549890 10395 rpc_server.cc:307] RPC server started. Bound to: 127.3.61.188:39275
I20260811 19:17:52.549944 10531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.61.188:39275 every 8 connection(s)
I20260811 19:17:52.551719 10395 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/minicluster-data/master-2/data/info.pb
I20260811 19:17:52.557057 10532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 19:17:52.561988 3318 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu as pid 10395
I20260811 19:17:52.562237 3318 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwEqXVd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786475454804547-3318-0/minicluster-data/master-2/wal/instance
I20260811 19:17:52.562795 3318 external_mini_cluster.cc:1428] Setting key 165d17b834b73d4639225e41d77228e6
I20260811 19:17:52.567592 10532 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.61.190" port: 44989 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 19:17:52 dist-test-slave-t7g5 krb5kdc[10098](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475870, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.188@KRBTEST.COM
Aug 11 19:17:52 dist-test-slave-t7g5 krb5kdc[10098](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475872, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.188@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:17:52.590157 10539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:52.572187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51874 (local address 127.3.61.190:44989)
0811 19:17:52.574165 (+ 1978us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:52.574170 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:52.574193 (+ 23us) server_negotiation.cc:407] Connection header received
0811 19:17:52.574260 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:52.574264 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:52.574351 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:52.574500 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:52.576294 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:52.577118 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:52.578908 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:52.579564 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:52.579726 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:52.584119 (+ 4393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:52.584127 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:52.584132 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:17:52.584187 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:52.587033 (+ 2846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:52.587860 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:52.587864 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:52.587866 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:52.588006 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:52.588396 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:52.588400 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:52.588402 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:52.588692 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:52.588837 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:52.589490 (+ 653us) server_negotiation.cc:299] Negotiation successful
0811 19:17:52.589780 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1856,"thread_start_us":1742,"threads_started":1}
I20260811 19:17:52.591476 10537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:52.565559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57784 (local address 127.3.61.188:39275)
0811 19:17:52.566788 (+ 1229us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:52.566808 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:52.566879 (+ 71us) server_negotiation.cc:407] Connection header received
0811 19:17:52.568861 (+ 1982us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:52.568911 (+ 50us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:52.569621 (+ 710us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:52.570215 (+ 594us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:52.572404 (+ 2189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:52.574811 (+ 2407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:52.576517 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:52.577407 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:52.577632 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:52.582853 (+ 5221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:52.582868 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:52.582898 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:17:52.582956 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:52.586469 (+ 3513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:52.587376 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:52.587384 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:52.587386 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:52.587573 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:52.587994 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:52.587998 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:52.588000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:52.588684 (+ 684us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:52.589031 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:52.589399 (+ 368us) server_negotiation.cc:299] Negotiation successful
0811 19:17:52.589892 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":875,"spinlock_wait_cycles":22784,"thread_start_us":477,"threads_started":1}
I20260811 19:17:52.591511 10538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:52.572041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.190:44989 (local address 127.0.0.1:51874)
0811 19:17:52.573280 (+ 1239us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:52.573358 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0811 19:17:52.573639 (+ 281us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:17:52.574808 (+ 1169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:17:52.574818 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:17:52.575732 (+ 914us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:17:52.576057 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:17:52.576070 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:17:52.577342 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:17:52.577348 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:17:52.578697 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:17:52.578712 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:17:52.579883 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:17:52.579888 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:17:52.580030 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:52.580854 (+ 824us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:17:52.580882 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:17:52.583801 (+ 2919us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:17:52.587199 (+ 3398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:17:52.587214 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:17:52.587229 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:17:52.587710 (+ 481us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:17:52.588125 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:17:52.588130 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:17:52.588133 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:17:52.588268 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:17:52.588844 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:17:52.588853 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:17:52.589088 (+ 235us) client_negotiation.cc:769] Sending connection context
0811 19:17:52.589325 (+ 237us) client_negotiation.cc:240] Negotiation successful
0811 19:17:52.590168 (+ 843us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":852,"mutex_wait_us":291,"thread_start_us":503,"threads_started":1}
I20260811 19:17:52.594157 10532 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.61.189" port: 45797 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 19:17:52 dist-test-slave-t7g5 krb5kdc[10098](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475872, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.188@KRBTEST.COM for kudu/127.3.61.189@KRBTEST.COM
Aug 11 19:17:52 dist-test-slave-t7g5 krb5kdc[10098](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475870, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.190@KRBTEST.COM
I20260811 19:17:52.611531 10538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:52.595046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.189:45797 (local address 127.0.0.1:58220)
0811 19:17:52.595450 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:52.595490 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 19:17:52.595614 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:17:52.596038 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:17:52.596043 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:17:52.596459 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:17:52.596627 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:17:52.596637 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:17:52.597948 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:17:52.597954 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:17:52.599095 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:17:52.599111 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:17:52.600189 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:17:52.600196 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:17:52.600450 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:52.601303 (+ 853us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:17:52.601323 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:17:52.603841 (+ 2518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:17:52.608041 (+ 4200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:17:52.608050 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:17:52.608054 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:17:52.608723 (+ 669us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:17:52.609385 (+ 662us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:17:52.609389 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:17:52.609392 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:17:52.609609 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:17:52.610348 (+ 739us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:17:52.610353 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:17:52.610549 (+ 196us) client_negotiation.cc:769] Sending connection context
0811 19:17:52.610693 (+ 144us) client_negotiation.cc:240] Negotiation successful
0811 19:17:52.611106 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178}
I20260811 19:17:52.611565 10540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:52.595135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58220 (local address 127.3.61.189:45797)
0811 19:17:52.595732 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:52.595738 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:52.595757 (+ 19us) server_negotiation.cc:407] Connection header received
0811 19:17:52.595808 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:52.595821 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:52.595882 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:52.595980 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:52.596800 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:52.597801 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:52.599298 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:52.599982 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:52.600139 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:52.604155 (+ 4016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:52.604164 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:52.604169 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:17:52.604225 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:52.607797 (+ 3572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:52.608953 (+ 1156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:52.608960 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:52.608963 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:52.609201 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:52.609789 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:52.609793 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:52.609796 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:52.610180 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:52.610350 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:52.610839 (+ 489us) server_negotiation.cc:299] Negotiation successful
0811 19:17:52.611138 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":363,"threads_started":1}
I20260811 19:17:52.613253 10532 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.61.188" port: 39275 } has no permanent_uuid. Determining permanent_uuid...
I20260811 19:17:52.615749 10539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:52.595162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51882 (local address 127.3.61.190:44989)
0811 19:17:52.595346 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:52.595352 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:52.595453 (+ 101us) server_negotiation.cc:407] Connection header received
0811 19:17:52.595775 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:52.595780 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:52.595874 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:52.595962 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:52.597496 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:52.598511 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:52.600312 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:52.600938 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:52.601043 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:52.607274 (+ 6231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:52.607283 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:52.607288 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:17:52.607350 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:52.610843 (+ 3493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:52.611882 (+ 1039us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:52.611888 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:52.611891 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:52.612073 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:52.612613 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:52.612617 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:52.612619 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:52.613046 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:52.613403 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:52.614941 (+ 1538us) server_negotiation.cc:299] Negotiation successful
0811 19:17:52.615404 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
Aug 11 19:17:52 dist-test-slave-t7g5 krb5kdc[10098](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475872, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.188@KRBTEST.COM for kudu/127.3.61.188@KRBTEST.COM
Aug 11 19:17:52 dist-test-slave-t7g5 krb5kdc[10098](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475870, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.61.189@KRBTEST.COM
I20260811 19:17:52.628652 10383 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.61.188" port: 39275 } attempt: 5
I20260811 19:17:52.630788 10538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:52.614673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:39275 (local address 127.0.0.1:57792)
0811 19:17:52.614960 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:52.614977 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 19:17:52.615115 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:17:52.615414 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:17:52.615418 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:17:52.615975 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:17:52.616173 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:17:52.616185 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:17:52.617492 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:17:52.617497 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:17:52.618999 (+ 1502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:17:52.619029 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:17:52.620128 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:17:52.620134 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:17:52.620328 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:52.620988 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:17:52.621007 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:17:52.623803 (+ 2796us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:17:52.627720 (+ 3917us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:17:52.627724 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:17:52.627727 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:17:52.628177 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:17:52.628618 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:17:52.628623 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:17:52.628625 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:17:52.628760 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:17:52.629488 (+ 728us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:17:52.629492 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:17:52.629683 (+ 191us) client_negotiation.cc:769] Sending connection context
0811 19:17:52.629856 (+ 173us) client_negotiation.cc:240] Negotiation successful
0811 19:17:52.630213 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219}
I20260811 19:17:52.630827 10537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:52.614669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57792 (local address 127.3.61.188:39275)
0811 19:17:52.614862 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:52.614868 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:52.615065 (+ 197us) server_negotiation.cc:407] Connection header received
0811 19:17:52.615225 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:52.615230 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:52.615289 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:52.615376 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:52.616328 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:52.617277 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:52.619220 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:52.619886 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:52.620070 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:52.624073 (+ 4003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:52.624081 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:52.624085 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:17:52.624130 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:52.627506 (+ 3376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:52.628354 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:52.628358 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:52.628360 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:52.628496 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:52.628878 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:52.628883 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:52.628886 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:52.629274 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:52.629434 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:52.630094 (+ 660us) server_negotiation.cc:299] Negotiation successful
0811 19:17:52.630370 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":68}
I20260811 19:17:52.637974 10540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:52.618233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58228 (local address 127.3.61.189:45797)
0811 19:17:52.618421 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:52.618428 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:52.618509 (+ 81us) server_negotiation.cc:407] Connection header received
0811 19:17:52.618691 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:52.618695 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:52.618764 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:52.618849 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:52.620583 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:52.621312 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:52.623043 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:52.623754 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:52.623945 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:52.629415 (+ 5470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:52.629422 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:52.629427 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:17:52.629473 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:52.633279 (+ 3806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:52.634296 (+ 1017us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:52.634301 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:52.634304 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:52.634736 (+ 432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:52.635504 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:52.635509 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:52.635511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:52.635985 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:52.636526 (+ 541us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:52.636880 (+ 354us) server_negotiation.cc:299] Negotiation successful
0811 19:17:52.637265 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 19:17:52.638414 10532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c1725bffbdb452fa4ce066f15ac5c38: Bootstrap starting.
Aug 11 19:17:52 dist-test-slave-t7g5 krb5kdc[10098](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475871, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.189@KRBTEST.COM for kudu/127.3.61.188@KRBTEST.COM
I20260811 19:17:52.643210 10532 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c1725bffbdb452fa4ce066f15ac5c38: Neither blocks nor log segments found. Creating new log.
I20260811 19:17:52.644601 3318 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 19:17:52.645272 10532 log.cc:824] T 00000000000000000000000000000000 P 3c1725bffbdb452fa4ce066f15ac5c38: Log is configured to *not* fsync() on all Append() calls
I20260811 19:17:52.651384 10386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:52.629387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:39275 (local address 127.0.0.1:57796)
0811 19:17:52.629723 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:52.629744 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 19:17:52.629871 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:17:52.631693 (+ 1822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:17:52.631700 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:17:52.632288 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:17:52.632540 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:17:52.632552 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:17:52.634047 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:17:52.634053 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:17:52.635702 (+ 1649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:17:52.635723 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:17:52.637270 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:17:52.637277 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:17:52.637612 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:52.639089 (+ 1477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:17:52.639121 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:17:52.642612 (+ 3491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:17:52.648152 (+ 5540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:17:52.648158 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:17:52.648161 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:17:52.648658 (+ 497us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:17:52.649242 (+ 584us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:17:52.649246 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:17:52.649249 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:17:52.649440 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:17:52.650205 (+ 765us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:17:52.650210 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:17:52.650423 (+ 213us) client_negotiation.cc:769] Sending connection context
0811 19:17:52.650592 (+ 169us) client_negotiation.cc:240] Negotiation successful
0811 19:17:52.650956 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174}
I20260811 19:17:52.647666 10198 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "38f4bd77341547e5bbd03a3d521b6b6d"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51882
I20260811 19:17:52.651911 10542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:52.629480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57796 (local address 127.3.61.188:39275)
0811 19:17:52.631353 (+ 1873us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:52.631359 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:52.631382 (+ 23us) server_negotiation.cc:407] Connection header received
0811 19:17:52.631434 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:52.631439 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:52.631513 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:52.631612 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:52.632772 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:52.633890 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:52.635937 (+ 2047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:52.637032 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:52.638723 (+ 1691us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:52.642902 (+ 4179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:52.642909 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:52.642914 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:17:52.642961 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:52.647918 (+ 4957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:52.648879 (+ 961us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:52.648885 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:52.648888 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:52.649085 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:52.649556 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:52.649561 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:52.649565 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:52.650060 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:52.650224 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:52.650966 (+ 742us) server_negotiation.cc:299] Negotiation successful
0811 19:17:52.651346 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1724,"thread_start_us":1488,"threads_started":1}
I20260811 19:17:52.652577 10532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c1725bffbdb452fa4ce066f15ac5c38: No bootstrap required, opened a new log
I20260811 19:17:52.657405 10383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdfac25e5f034b68b2f140192370d2b4: Bootstrap starting.
I20260811 19:17:52.662147 10383 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bdfac25e5f034b68b2f140192370d2b4: Neither blocks nor log segments found. Creating new log.
I20260811 19:17:52.662428 10236 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.61.188" port: 39275 } attempt: 5
I20260811 19:17:52.664076 10383 log.cc:824] T 00000000000000000000000000000000 P bdfac25e5f034b68b2f140192370d2b4: Log is configured to *not* fsync() on all Append() calls
I20260811 19:17:52.663880 10532 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c1725bffbdb452fa4ce066f15ac5c38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38f4bd77341547e5bbd03a3d521b6b6d" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 44989 } } peers { permanent_uuid: "bdfac25e5f034b68b2f140192370d2b4" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 45797 } } peers { permanent_uuid: "3c1725bffbdb452fa4ce066f15ac5c38" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 39275 } }
I20260811 19:17:52.664502 10532 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c1725bffbdb452fa4ce066f15ac5c38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c1725bffbdb452fa4ce066f15ac5c38, State: Initialized, Role: FOLLOWER
I20260811 19:17:52.665131 10532 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c1725bffbdb452fa4ce066f15ac5c38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38f4bd77341547e5bbd03a3d521b6b6d" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 44989 } } peers { permanent_uuid: "bdfac25e5f034b68b2f140192370d2b4" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 45797 } } peers { permanent_uuid: "3c1725bffbdb452fa4ce066f15ac5c38" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 39275 } }
I20260811 19:17:52.666563 10550 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c1725bffbdb452fa4ce066f15ac5c38 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38f4bd77341547e5bbd03a3d521b6b6d" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 44989 } } peers { permanent_uuid: "bdfac25e5f034b68b2f140192370d2b4" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 45797 } } peers { permanent_uuid: "3c1725bffbdb452fa4ce066f15ac5c38" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 39275 } } }
I20260811 19:17:52.667654 10550 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c1725bffbdb452fa4ce066f15ac5c38 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:17:52.667778 10532 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c1725bffbdb452fa4ce066f15ac5c38 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:17:52.668151 10383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdfac25e5f034b68b2f140192370d2b4: No bootstrap required, opened a new log
Aug 11 19:17:52 dist-test-slave-t7g5 krb5kdc[10098](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786475871, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.61.190@KRBTEST.COM for kudu/127.3.61.188@KRBTEST.COM
I20260811 19:17:52.678469 10383 raft_consensus.cc:348] T 00000000000000000000000000000000 P bdfac25e5f034b68b2f140192370d2b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38f4bd77341547e5bbd03a3d521b6b6d" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 44989 } } peers { permanent_uuid: "bdfac25e5f034b68b2f140192370d2b4" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 45797 } } peers { permanent_uuid: "3c1725bffbdb452fa4ce066f15ac5c38" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 39275 } }
I20260811 19:17:52.678936 10383 raft_consensus.cc:729] T 00000000000000000000000000000000 P bdfac25e5f034b68b2f140192370d2b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdfac25e5f034b68b2f140192370d2b4, State: Initialized, Role: FOLLOWER
I20260811 19:17:52.679587 10383 consensus_queue.cc:260] T 00000000000000000000000000000000 P bdfac25e5f034b68b2f140192370d2b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38f4bd77341547e5bbd03a3d521b6b6d" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 44989 } } peers { permanent_uuid: "bdfac25e5f034b68b2f140192370d2b4" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 45797 } } peers { permanent_uuid: "3c1725bffbdb452fa4ce066f15ac5c38" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 39275 } }
I20260811 19:17:52.681612 10559 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdfac25e5f034b68b2f140192370d2b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38f4bd77341547e5bbd03a3d521b6b6d" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 44989 } } peers { permanent_uuid: "bdfac25e5f034b68b2f140192370d2b4" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 45797 } } peers { permanent_uuid: "3c1725bffbdb452fa4ce066f15ac5c38" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 39275 } } }
I20260811 19:17:52.682749 10383 sys_catalog.cc:564] T 00000000000000000000000000000000 P bdfac25e5f034b68b2f140192370d2b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:17:52.683281 10559 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdfac25e5f034b68b2f140192370d2b4 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:17:52.684954 10241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:52.663279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.61.188:39275 (local address 127.0.0.1:57804)
0811 19:17:52.663505 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0811 19:17:52.663527 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 19:17:52.663672 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 19:17:52.664933 (+ 1261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 19:17:52.664939 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 19:17:52.665509 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0811 19:17:52.665791 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:17:52.665808 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:17:52.667494 (+ 1686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:17:52.667500 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:17:52.668916 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 19:17:52.668952 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 19:17:52.670293 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 19:17:52.670300 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 19:17:52.670534 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:52.671290 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 19:17:52.671319 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 19:17:52.674721 (+ 3402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 19:17:52.680882 (+ 6161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:17:52.680889 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:17:52.680893 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:17:52.681533 (+ 640us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:17:52.682138 (+ 605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 19:17:52.682143 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 19:17:52.682147 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 19:17:52.682434 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 19:17:52.683242 (+ 808us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 19:17:52.683247 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 19:17:52.683700 (+ 453us) client_negotiation.cc:769] Sending connection context
0811 19:17:52.683935 (+ 235us) client_negotiation.cc:240] Negotiation successful
0811 19:17:52.684506 (+ 571us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260811 19:17:52.686617 10542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:52.663355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57804 (local address 127.3.61.188:39275)
0811 19:17:52.664518 (+ 1163us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:52.664524 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:52.664547 (+ 23us) server_negotiation.cc:407] Connection header received
0811 19:17:52.664614 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:52.664619 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:52.664729 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:52.665984 (+ 1255us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:52.666138 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:52.667286 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:52.669199 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:52.670085 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:52.670246 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:52.675264 (+ 5018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:52.675275 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:52.675306 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:17:52.675402 (+ 96us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:52.680629 (+ 5227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:52.681793 (+ 1164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:52.681798 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:52.681801 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:52.681983 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:52.682609 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:52.682614 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:52.682617 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:52.683089 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:52.685330 (+ 2241us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:52.685596 (+ 266us) server_negotiation.cc:299] Negotiation successful
0811 19:17:52.686029 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":981}
I20260811 19:17:52.693840 10236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38f4bd77341547e5bbd03a3d521b6b6d: Bootstrap starting.
I20260811 19:17:52.704586 10236 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 38f4bd77341547e5bbd03a3d521b6b6d: Neither blocks nor log segments found. Creating new log.
I20260811 19:17:52.707979 10236 log.cc:824] T 00000000000000000000000000000000 P 38f4bd77341547e5bbd03a3d521b6b6d: Log is configured to *not* fsync() on all Append() calls
I20260811 19:17:52.700217 10250 master_runner.cc:186] Error getting master registration for 127.3.61.190:44989: OK, instance_id { permanent_uuid: "38f4bd77341547e5bbd03a3d521b6b6d" instance_seqno: 1786475871214485 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 19:17:52.713621 10250 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 19:17:52.717046 10583 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bdfac25e5f034b68b2f140192370d2b4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 19:17:52.717228 10583 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 19:17:52.720244 10236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38f4bd77341547e5bbd03a3d521b6b6d: No bootstrap required, opened a new log
I20260811 19:17:52.720727 10395 master_runner.cc:186] Error getting master registration for 127.3.61.190:44989: OK, instance_id { permanent_uuid: "38f4bd77341547e5bbd03a3d521b6b6d" instance_seqno: 1786475871214485 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 19:17:52.733687 10395 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 19:17:52.734794 10236 raft_consensus.cc:348] T 00000000000000000000000000000000 P 38f4bd77341547e5bbd03a3d521b6b6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38f4bd77341547e5bbd03a3d521b6b6d" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 44989 } } peers { permanent_uuid: "bdfac25e5f034b68b2f140192370d2b4" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 45797 } } peers { permanent_uuid: "3c1725bffbdb452fa4ce066f15ac5c38" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 39275 } }
W20260811 19:17:52.736403 10593 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3c1725bffbdb452fa4ce066f15ac5c38: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 19:17:52.736600 10593 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 19:17:52.737839 10236 raft_consensus.cc:729] T 00000000000000000000000000000000 P 38f4bd77341547e5bbd03a3d521b6b6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38f4bd77341547e5bbd03a3d521b6b6d, State: Initialized, Role: FOLLOWER
I20260811 19:17:52.738574 10236 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38f4bd77341547e5bbd03a3d521b6b6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38f4bd77341547e5bbd03a3d521b6b6d" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 44989 } } peers { permanent_uuid: "bdfac25e5f034b68b2f140192370d2b4" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 45797 } } peers { permanent_uuid: "3c1725bffbdb452fa4ce066f15ac5c38" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 39275 } }
I20260811 19:17:52.739776 10595 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38f4bd77341547e5bbd03a3d521b6b6d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38f4bd77341547e5bbd03a3d521b6b6d" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 44989 } } peers { permanent_uuid: "bdfac25e5f034b68b2f140192370d2b4" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 45797 } } peers { permanent_uuid: "3c1725bffbdb452fa4ce066f15ac5c38" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 39275 } } }
I20260811 19:17:52.740571 10595 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38f4bd77341547e5bbd03a3d521b6b6d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:17:52.748358 10236 sys_catalog.cc:564] T 00000000000000000000000000000000 P 38f4bd77341547e5bbd03a3d521b6b6d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 19:17:52.748548 10198 raft_consensus.cc:482] T 00000000000000000000000000000000 P 38f4bd77341547e5bbd03a3d521b6b6d [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 19:17:52.748891 10198 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 38f4bd77341547e5bbd03a3d521b6b6d [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:17:52.753140 10198 raft_consensus.cc:504] T 00000000000000000000000000000000 P 38f4bd77341547e5bbd03a3d521b6b6d [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38f4bd77341547e5bbd03a3d521b6b6d" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 44989 } } peers { permanent_uuid: "bdfac25e5f034b68b2f140192370d2b4" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 45797 } } peers { permanent_uuid: "3c1725bffbdb452fa4ce066f15ac5c38" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 39275 } }
I20260811 19:17:52.756341 10331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "38f4bd77341547e5bbd03a3d521b6b6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "bdfac25e5f034b68b2f140192370d2b4"
I20260811 19:17:52.756956 10331 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bdfac25e5f034b68b2f140192370d2b4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:17:52.760934 10331 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bdfac25e5f034b68b2f140192370d2b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38f4bd77341547e5bbd03a3d521b6b6d in term 1.
I20260811 19:17:52.762024 10156 leader_election.cc:302] T 00000000000000000000000000000000 P 38f4bd77341547e5bbd03a3d521b6b6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38f4bd77341547e5bbd03a3d521b6b6d, bdfac25e5f034b68b2f140192370d2b4; no voters:
I20260811 19:17:52.762748 10595 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 38f4bd77341547e5bbd03a3d521b6b6d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 19:17:52.770438 10198 leader_election.cc:288] T 00000000000000000000000000000000 P 38f4bd77341547e5bbd03a3d521b6b6d [CANDIDATE]: Term 1 election: Requested vote from peers bdfac25e5f034b68b2f140192370d2b4 (127.3.61.189:45797), 3c1725bffbdb452fa4ce066f15ac5c38 (127.3.61.188:39275)
I20260811 19:17:52.770648 10484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "38f4bd77341547e5bbd03a3d521b6b6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "3c1725bffbdb452fa4ce066f15ac5c38"
I20260811 19:17:52.771319 10484 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c1725bffbdb452fa4ce066f15ac5c38 [term 0 FOLLOWER]: Advancing to term 1
I20260811 19:17:52.776036 10484 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3c1725bffbdb452fa4ce066f15ac5c38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38f4bd77341547e5bbd03a3d521b6b6d in term 1.
W20260811 19:17:52.777981 10452 tablet.cc:2369] T 00000000000000000000000000000000 P 3c1725bffbdb452fa4ce066f15ac5c38: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:17:52.788061 10595 raft_consensus.cc:686] T 00000000000000000000000000000000 P 38f4bd77341547e5bbd03a3d521b6b6d [term 1 LEADER]: Becoming Leader. State: Replica: 38f4bd77341547e5bbd03a3d521b6b6d, State: Running, Role: LEADER
I20260811 19:17:52.789325 10595 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38f4bd77341547e5bbd03a3d521b6b6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38f4bd77341547e5bbd03a3d521b6b6d" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 44989 } } peers { permanent_uuid: "bdfac25e5f034b68b2f140192370d2b4" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 45797 } } peers { permanent_uuid: "3c1725bffbdb452fa4ce066f15ac5c38" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 39275 } }
I20260811 19:17:52.802223 10599 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38f4bd77341547e5bbd03a3d521b6b6d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38f4bd77341547e5bbd03a3d521b6b6d. Latest consensus state: current_term: 1 leader_uuid: "38f4bd77341547e5bbd03a3d521b6b6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38f4bd77341547e5bbd03a3d521b6b6d" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 44989 } } peers { permanent_uuid: "bdfac25e5f034b68b2f140192370d2b4" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 45797 } } peers { permanent_uuid: "3c1725bffbdb452fa4ce066f15ac5c38" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 39275 } } }
I20260811 19:17:52.802871 10599 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38f4bd77341547e5bbd03a3d521b6b6d [sys.catalog]: This master's current role is: LEADER
I20260811 19:17:52.807964 10615 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 19:17:52.841015 10615 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260811 19:17:52.858075 10291 tablet.cc:2369] T 00000000000000000000000000000000 P bdfac25e5f034b68b2f140192370d2b4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 19:17:52.864203 10540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:52.812650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58240 (local address 127.3.61.189:45797)
0811 19:17:52.812931 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:52.812940 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:52.822438 (+ 9498us) server_negotiation.cc:407] Connection header received
0811 19:17:52.822761 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:52.822767 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:52.822885 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:52.822972 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:52.825698 (+ 2726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:52.826793 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:52.829817 (+ 3024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:52.830501 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:52.830623 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:52.848862 (+ 18239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:52.848872 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:52.848878 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:17:52.848957 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:52.853759 (+ 4802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:52.857224 (+ 3465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:52.857231 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:52.857235 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:52.857507 (+ 272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:52.862011 (+ 4504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:52.862019 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:52.862022 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:52.862517 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:52.862840 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:52.863529 (+ 689us) server_negotiation.cc:299] Negotiation successful
0811 19:17:52.863804 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 19:17:52.869642 10539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:52.817405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51894 (local address 127.3.61.190:44989)
0811 19:17:52.817616 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:52.817623 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:52.823567 (+ 5944us) server_negotiation.cc:407] Connection header received
0811 19:17:52.825375 (+ 1808us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:52.825382 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:52.825490 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:52.825569 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:52.833816 (+ 8247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:52.835285 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:52.842226 (+ 6941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:52.843022 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:52.843152 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:52.859354 (+ 16202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:52.859364 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:52.859370 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:17:52.859427 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:52.863327 (+ 3900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:52.864207 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:52.864212 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:52.864215 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:52.864569 (+ 354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:52.867209 (+ 2640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:52.867215 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:52.867218 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:52.867713 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:52.868139 (+ 426us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:52.868828 (+ 689us) server_negotiation.cc:299] Negotiation successful
0811 19:17:52.869241 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 19:17:52.872825 10542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:52.813821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57818 (local address 127.3.61.188:39275)
0811 19:17:52.814011 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:52.814017 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:52.822687 (+ 8670us) server_negotiation.cc:407] Connection header received
0811 19:17:52.823731 (+ 1044us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:52.823738 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:52.823831 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:52.823914 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:52.845183 (+ 21269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:52.846478 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:52.854887 (+ 8409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:52.855754 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:52.855907 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:52.865279 (+ 9372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 19:17:52.865287 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 19:17:52.865292 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 19:17:52.865343 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 19:17:52.869131 (+ 3788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:52.870185 (+ 1054us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:52.870191 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:52.870194 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:52.870418 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 19:17:52.870874 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 19:17:52.870879 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 19:17:52.870881 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 19:17:52.871265 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 19:17:52.871564 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:52.872142 (+ 578us) server_negotiation.cc:299] Negotiation successful
0811 19:17:52.872419 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 19:17:52.875902 10639 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 19:17:52.888247 10484 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3c1725bffbdb452fa4ce066f15ac5c38 [term 1 FOLLOWER]: Refusing update from remote peer 38f4bd77341547e5bbd03a3d521b6b6d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:17:52.889971 10331 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bdfac25e5f034b68b2f140192370d2b4 [term 1 FOLLOWER]: Refusing update from remote peer 38f4bd77341547e5bbd03a3d521b6b6d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 19:17:52.891026 10599 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 38f4bd77341547e5bbd03a3d521b6b6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c1725bffbdb452fa4ce066f15ac5c38" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 39275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260811 19:17:52.891844 10595 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 38f4bd77341547e5bbd03a3d521b6b6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdfac25e5f034b68b2f140192370d2b4" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 45797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 19:17:52.921345 10559 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdfac25e5f034b68b2f140192370d2b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38f4bd77341547e5bbd03a3d521b6b6d. Latest consensus state: current_term: 1 leader_uuid: "38f4bd77341547e5bbd03a3d521b6b6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38f4bd77341547e5bbd03a3d521b6b6d" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 44989 } } peers { permanent_uuid: "bdfac25e5f034b68b2f140192370d2b4" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 45797 } } peers { permanent_uuid: "3c1725bffbdb452fa4ce066f15ac5c38" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 39275 } } }
I20260811 19:17:52.922062 10559 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdfac25e5f034b68b2f140192370d2b4 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:17:52.929612 10634 mvcc.cc:205] Tried to move back new op lower bound from 7317405175319601152 to 7317405174967451648. Current Snapshot: MvccSnapshot[applied={T|T < 7317405175319601152}]
I20260811 19:17:52.931324 10615 catalog_manager.cc:1347] Generated new cluster ID: 46fa2c90763d49a5aa21029ca8247e08
I20260811 19:17:52.931468 10615 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 19:17:52.932484 10550 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c1725bffbdb452fa4ce066f15ac5c38 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38f4bd77341547e5bbd03a3d521b6b6d. Latest consensus state: current_term: 1 leader_uuid: "38f4bd77341547e5bbd03a3d521b6b6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38f4bd77341547e5bbd03a3d521b6b6d" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 44989 } } peers { permanent_uuid: "bdfac25e5f034b68b2f140192370d2b4" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 45797 } } peers { permanent_uuid: "3c1725bffbdb452fa4ce066f15ac5c38" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 39275 } } }
I20260811 19:17:52.933066 10550 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c1725bffbdb452fa4ce066f15ac5c38 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:17:52.936642 10595 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38f4bd77341547e5bbd03a3d521b6b6d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "38f4bd77341547e5bbd03a3d521b6b6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38f4bd77341547e5bbd03a3d521b6b6d" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 44989 } } peers { permanent_uuid: "bdfac25e5f034b68b2f140192370d2b4" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 45797 } } peers { permanent_uuid: "3c1725bffbdb452fa4ce066f15ac5c38" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 39275 } } }
I20260811 19:17:52.937072 10595 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38f4bd77341547e5bbd03a3d521b6b6d [sys.catalog]: This master's current role is: LEADER
I20260811 19:17:52.937393 10595 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38f4bd77341547e5bbd03a3d521b6b6d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "38f4bd77341547e5bbd03a3d521b6b6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38f4bd77341547e5bbd03a3d521b6b6d" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 44989 } } peers { permanent_uuid: "bdfac25e5f034b68b2f140192370d2b4" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 45797 } } peers { permanent_uuid: "3c1725bffbdb452fa4ce066f15ac5c38" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 39275 } } }
I20260811 19:17:52.937733 10595 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38f4bd77341547e5bbd03a3d521b6b6d [sys.catalog]: This master's current role is: LEADER
I20260811 19:17:52.938406 10642 mvcc.cc:205] Tried to move back new op lower bound from 7317405175319601152 to 7317405174967451648. Current Snapshot: MvccSnapshot[applied={T|T < 7317405175319601152}]
I20260811 19:17:52.943642 10652 mvcc.cc:205] Tried to move back new op lower bound from 7317405175319601152 to 7317405174967451648. Current Snapshot: MvccSnapshot[applied={T|T < 7317405175319601152}]
I20260811 19:17:52.960146 10559 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdfac25e5f034b68b2f140192370d2b4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "38f4bd77341547e5bbd03a3d521b6b6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38f4bd77341547e5bbd03a3d521b6b6d" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 44989 } } peers { permanent_uuid: "bdfac25e5f034b68b2f140192370d2b4" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 45797 } } peers { permanent_uuid: "3c1725bffbdb452fa4ce066f15ac5c38" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 39275 } } }
I20260811 19:17:52.960829 10559 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdfac25e5f034b68b2f140192370d2b4 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:17:52.968608 10615 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 19:17:52.971623 10615 catalog_manager.cc:1504] Loading token signing keys...
I20260811 19:17:52.984207 10550 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c1725bffbdb452fa4ce066f15ac5c38 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "38f4bd77341547e5bbd03a3d521b6b6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38f4bd77341547e5bbd03a3d521b6b6d" member_type: VOTER last_known_addr { host: "127.3.61.190" port: 44989 } } peers { permanent_uuid: "bdfac25e5f034b68b2f140192370d2b4" member_type: VOTER last_known_addr { host: "127.3.61.189" port: 45797 } } peers { permanent_uuid: "3c1725bffbdb452fa4ce066f15ac5c38" member_type: VOTER last_known_addr { host: "127.3.61.188" port: 39275 } } }
I20260811 19:17:52.984566 10550 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c1725bffbdb452fa4ce066f15ac5c38 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 19:17:53.004302 10615 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 38f4bd77341547e5bbd03a3d521b6b6d: Generated new TSK 0
I20260811 19:17:53.010021 10615 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 19:17:53.129432 3318 mini_kdc.cc:348] Time spent kdestroy: real 0.102s user 0.002s sys 0.037s
W20260811 19:17:53.192718 10540 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:17:53.168011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58242 (local address 127.3.61.189:45797)
0811 19:17:53.168274 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:53.168281 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:53.188656 (+ 20375us) server_negotiation.cc:407] Connection header received
0811 19:17:53.190986 (+ 2330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:53.190994 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:53.191093 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:53.191169 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:53.192440 (+ 1271us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58242: BlockingRecv error: recv got EOF from 127.0.0.1:58242 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260811 19:17:53.195778 10542 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:17:53.174719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57824 (local address 127.3.61.188:39275)
0811 19:17:53.174949 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:53.174956 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:53.190600 (+ 15644us) server_negotiation.cc:407] Connection header received
0811 19:17:53.193250 (+ 2650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:53.193258 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:53.193359 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:53.193453 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:53.195511 (+ 2058us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57824: BlockingRecv error: recv got EOF from 127.0.0.1:57824 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
W20260811 19:17:53.199322 10539 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:17:53.186095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51900 (local address 127.3.61.190:44989)
0811 19:17:53.186373 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:53.186380 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:53.194686 (+ 8306us) server_negotiation.cc:407] Connection header received
0811 19:17:53.197046 (+ 2360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:53.197054 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:53.197147 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:53.197222 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:53.198948 (+ 1726us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51900: BlockingRecv error: recv got EOF from 127.0.0.1:51900 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260811 19:17:53.246713 10540 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:17:53.233109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58248 (local address 127.3.61.189:45797)
0811 19:17:53.233300 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:53.233306 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:53.244302 (+ 10996us) server_negotiation.cc:407] Connection header received
0811 19:17:53.244893 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:53.244900 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:53.245064 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:53.245138 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:53.246434 (+ 1296us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58248: BlockingRecv error: recv got EOF from 127.0.0.1:58248 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260811 19:17:53.295871 10540 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:17:53.286439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58254 (local address 127.3.61.189:45797)
0811 19:17:53.286664 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:53.286672 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:53.293604 (+ 6932us) server_negotiation.cc:407] Connection header received
0811 19:17:53.294097 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:53.294104 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:53.294216 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:53.294308 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:53.295593 (+ 1285us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58254: BlockingRecv error: recv got EOF from 127.0.0.1:58254 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
W20260811 19:17:53.345029 10540 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:17:53.331172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58270 (local address 127.3.61.189:45797)
0811 19:17:53.331401 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:53.331407 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:53.342515 (+ 11108us) server_negotiation.cc:407] Connection header received
0811 19:17:53.343188 (+ 673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:53.343196 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:53.343328 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:53.343425 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:53.344682 (+ 1257us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58270: BlockingRecv error: recv got EOF from 127.0.0.1:58270 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
W20260811 19:17:53.421710 10540 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:17:53.402964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58278 (local address 127.3.61.189:45797)
0811 19:17:53.403477 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:53.403487 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:53.418794 (+ 15307us) server_negotiation.cc:407] Connection header received
0811 19:17:53.419135 (+ 341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:53.419147 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:53.419300 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:53.419453 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:53.421400 (+ 1947us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58278: BlockingRecv error: recv got EOF from 127.0.0.1:58278 (error 108)
Metrics: {"server-negotiator.queue_time_us":283}
W20260811 19:17:53.446133 10540 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:17:53.443542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58294 (local address 127.3.61.189:45797)
0811 19:17:53.443790 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:53.443798 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:53.444308 (+ 510us) server_negotiation.cc:407] Connection header received
0811 19:17:53.444753 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:53.444763 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:53.444863 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:53.444961 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:53.445847 (+ 886us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58294: BlockingRecv error: recv got EOF from 127.0.0.1:58294 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
W20260811 19:17:53.469309 10540 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:17:53.466668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58308 (local address 127.3.61.189:45797)
0811 19:17:53.466944 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:53.466952 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:53.467541 (+ 589us) server_negotiation.cc:407] Connection header received
0811 19:17:53.467784 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:53.467788 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:53.467896 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:53.468007 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:53.468973 (+ 966us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58308: BlockingRecv error: recv got EOF from 127.0.0.1:58308 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
W20260811 19:17:53.508931 10540 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:17:53.506446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58310 (local address 127.3.61.189:45797)
0811 19:17:53.506716 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:53.506725 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:53.507052 (+ 327us) server_negotiation.cc:407] Connection header received
0811 19:17:53.507267 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:53.507274 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:53.507399 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:53.507568 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:53.508626 (+ 1058us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58310: BlockingRecv error: recv got EOF from 127.0.0.1:58310 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
W20260811 19:17:53.580137 10540 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:17:53.577024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58324 (local address 127.3.61.189:45797)
0811 19:17:53.577288 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:53.577296 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:53.578152 (+ 856us) server_negotiation.cc:407] Connection header received
0811 19:17:53.578578 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:53.578585 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:53.578729 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:53.578894 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:53.579899 (+ 1005us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58324: BlockingRecv error: recv got EOF from 127.0.0.1:58324 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
W20260811 19:17:53.714418 10540 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 19:17:53.711908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53166 (local address 127.3.61.189:45797)
0811 19:17:53.712109 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:53.712114 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:53.712467 (+ 353us) server_negotiation.cc:407] Connection header received
0811 19:17:53.712674 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:53.712678 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:53.712758 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:53.712860 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 19:17:53.714089 (+ 1229us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53166: BlockingRecv error: recv got EOF from 127.0.0.1:53166 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 19:17:53.719826 10583 catalog_manager.cc:1259] Loaded cluster ID: 46fa2c90763d49a5aa21029ca8247e08
I20260811 19:17:53.719942 10583 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bdfac25e5f034b68b2f140192370d2b4: loading cluster ID for follower catalog manager: success
I20260811 19:17:53.723194 10583 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bdfac25e5f034b68b2f140192370d2b4: acquiring CA information for follower catalog manager: success
I20260811 19:17:53.725569 10583 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bdfac25e5f034b68b2f140192370d2b4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 19:17:53.738920 10593 catalog_manager.cc:1259] Loaded cluster ID: 46fa2c90763d49a5aa21029ca8247e08
I20260811 19:17:53.739041 10593 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3c1725bffbdb452fa4ce066f15ac5c38: loading cluster ID for follower catalog manager: success
I20260811 19:17:53.741711 10593 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3c1725bffbdb452fa4ce066f15ac5c38: acquiring CA information for follower catalog manager: success
I20260811 19:17:53.743686 10593 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3c1725bffbdb452fa4ce066f15ac5c38: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 19:17:53.981621 10540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:53.974235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53182 (local address 127.3.61.189:45797)
0811 19:17:53.974651 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:53.974660 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:53.974904 (+ 244us) server_negotiation.cc:407] Connection header received
0811 19:17:53.975173 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:53.975178 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:53.975412 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:53.975596 (+ 184us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:17:53.976200 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:53.977008 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:53.978668 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:53.979099 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:53.979227 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:53.980179 (+ 952us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:17:53.980795 (+ 616us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:17:53.980938 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:53.981195 (+ 257us) server_negotiation.cc:299] Negotiation successful
0811 19:17:53.981292 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154}
I20260811 19:17:53.996431 10785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:53.989302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33106 (local address 127.3.61.188:39275)
0811 19:17:53.990036 (+ 734us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:53.990052 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:53.990180 (+ 128us) server_negotiation.cc:407] Connection header received
0811 19:17:53.990475 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:53.990480 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:53.990645 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:53.990824 (+ 179us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:17:53.991285 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:53.992055 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:53.993745 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:53.994366 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:53.994497 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:53.995078 (+ 581us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:17:53.995713 (+ 635us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:17:53.995866 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:53.995994 (+ 128us) server_negotiation.cc:299] Negotiation successful
0811 19:17:53.996089 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":380,"threads_started":1}
I20260811 19:17:54.017750 10540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:54.003482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53186 (local address 127.3.61.189:45797)
0811 19:17:54.003663 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:54.003670 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:54.011032 (+ 7362us) server_negotiation.cc:407] Connection header received
0811 19:17:54.011291 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:54.011297 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:54.011416 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:54.011515 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:17:54.012341 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:54.013341 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:54.015591 (+ 2250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:54.016178 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:54.016322 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:54.016825 (+ 503us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:17:54.016990 (+ 165us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:17:54.017070 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:54.017297 (+ 227us) server_negotiation.cc:299] Negotiation successful
0811 19:17:54.017409 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 19:17:54.017897 10785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:54.003205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33114 (local address 127.3.61.188:39275)
0811 19:17:54.003664 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:54.003670 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:54.011162 (+ 7492us) server_negotiation.cc:407] Connection header received
0811 19:17:54.011458 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:54.011465 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:54.011556 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:54.011628 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:17:54.012458 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:54.013393 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:54.015799 (+ 2406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:54.016455 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:54.016568 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:54.017089 (+ 521us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:17:54.017217 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:17:54.017278 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:54.017500 (+ 222us) server_negotiation.cc:299] Negotiation successful
0811 19:17:54.017589 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 19:17:54.019044 10805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 19:17:54.003084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46524 (local address 127.3.61.190:44989)
0811 19:17:54.009870 (+ 6786us) server_negotiation.cc:206] Beginning negotiation
0811 19:17:54.009880 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 19:17:54.011829 (+ 1949us) server_negotiation.cc:407] Connection header received
0811 19:17:54.012100 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 19:17:54.012104 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 19:17:54.012284 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 19:17:54.012495 (+ 211us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 19:17:54.013024 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:54.014109 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:54.016289 (+ 2180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 19:17:54.016892 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 19:17:54.017002 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 19:17:54.017543 (+ 541us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 19:17:54.018074 (+ 531us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 19:17:54.018181 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 19:17:54.018628 (+ 447us) server_negotiation.cc:299] Negotiation successful
0811 19:17:54.018720 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6613,"thread_start_us":6669,"threads_started":1}
I20260811 19:17:54.025391 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 10123
I20260811 19:17:54.027551 10229 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:17:54.145097 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 10123
I20260811 19:17:54.181198 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 10250
W20260811 19:17:54.181216 10289 connection.cc:535] server connection from 127.0.0.1:58208 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 19:17:54.182698 10370 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:17:54.304131 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 10250
I20260811 19:17:54.342015 3318 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu pid 10395
I20260811 19:17:54.343189 10522 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 19:17:54.470041 3318 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwEqXVd/build/asan/bin/kudu with pid 10395
2026-08-11T19:17:54Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (5647 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (11091 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (419576 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST